-- Logs begin at Wed 2022-08-24 08:14:00 UTC, end at Wed 2022-08-24 09:17:52 UTC. -- Aug 24 08:40:44.472423 dv-cinder-835384-23-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 08:40:44.481191 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 08:40:44.483436 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 08:40:44 2022] *** Aug 24 08:40:44.483436 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 08:40:44.483436 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 08:40:44.483436 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: nodename: dv-cinder-835384-23-iscsi Aug 24 08:40:44.483436 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: machine: x86_64 Aug 24 08:40:44.483436 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: clock source: unix Aug 24 08:40:44.483436 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: pcre jit disabled Aug 24 08:40:44.483436 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: detected number of CPU cores: 4 Aug 24 08:40:44.483941 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: current working directory: / Aug 24 08:40:44.483941 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: detected binary path: /usr/bin/uwsgi-core Aug 24 08:40:44.483941 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: your processes number limit is 47680 Aug 24 08:40:44.483941 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: your memory page size is 4096 bytes Aug 24 08:40:44.483941 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: detected max file descriptor number: 2048 Aug 24 08:40:44.483941 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: lock engine: pthread robust mutexes Aug 24 08:40:44.483941 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: thunder lock: enabled Aug 24 08:40:44.483941 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 08:40:44.483941 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 08:40:44.525947 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: Python main interpreter initialized at 0x5600e0e8e9c0 Aug 24 08:40:44.525947 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: python threads support enabled Aug 24 08:40:44.525947 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: your server socket listen backlog is limited to 100 connections Aug 24 08:40:44.525947 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: your mercy for graceful operations on workers is 90 seconds Aug 24 08:40:44.526408 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 08:40:44.526408 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: *** Operational MODE: preforking *** Aug 24 08:40:44.526695 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: *** uWSGI is running in multiple interpreter mode *** Aug 24 08:40:44.526695 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: spawned uWSGI master process (pid: 81460) Aug 24 08:40:44.526945 dv-cinder-835384-23-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 08:40:44.528604 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: spawned uWSGI worker 1 (pid: 81475, cores: 1) Aug 24 08:40:44.528604 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: spawned uWSGI worker 2 (pid: 81476, cores: 1) Aug 24 08:40:44.528604 dv-cinder-835384-23-iscsi devstack@keystone.service[81460]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 08:40:48.907509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 08:40:48.907509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: from cryptography.utils import int_from_bytes Aug 24 08:40:48.908335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 08:40:48.908335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: from cryptography.utils import int_from_bytes Aug 24 08:40:48.909130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 08:40:48.909130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: from cryptography.utils import int_from_bytes Aug 24 08:40:48.909599 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 08:40:48.909599 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: from cryptography.utils import int_from_bytes Aug 24 08:40:48.959247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 08:40:48.959471 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 08:40:48.959622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] command line args: [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 08:40:48.959891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 08:40:48.960072 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ================================================================================ {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 08:40:48.960277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 08:40:48.960364 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] admin_token = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.960476 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 08:40:48.960594 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] config_dir = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.960675 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] command line args: [] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 08:40:48.960782 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.960929 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 08:40:48.961022 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] config_source = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.961100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ================================================================================ {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 08:40:48.961181 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] debug = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.961321 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.961580 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] admin_token = **** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.961580 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] config_dir = [] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.961700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.961766 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.961905 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.962011 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] config_source = [] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.962105 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] debug = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.962204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.962307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.962432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.962535 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.962630 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.962722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.962821 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_config_append = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.963432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.963549 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.963646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_dir = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.963744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.963846 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_file = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.963945 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] list_limit = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.964056 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.964202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] log_config_append = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.964326 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.964407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.964530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] log_dir = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.964611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.964737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] log_file = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.964819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.964896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.965019 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.965098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.965224 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.965307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.965386 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.965508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.965592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.965692 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.965810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.965890 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.966047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.966170 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.966283 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.966368 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.966519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.966652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.966769 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.966849 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.967002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.967081 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.967241 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.967321 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.967488 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.967735 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.967812 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.67/identity {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.967941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] publish_errors = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.968017 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.968198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.968433 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.968554 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.67/identity {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.968642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.968714 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] publish_errors = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.968852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.968940 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.969092 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.969171 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.969290 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.969368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] standard_threads = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.969539 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.969619 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.969776 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.969863 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.969941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.970012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] standard_threads = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.970147 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] use_journal = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.970228 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.970306 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] use_json = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.970454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] use_stderr = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.970581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.970663 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.970749 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] use_syslog = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.970829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] use_journal = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.970907 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.970984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] use_json = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.971099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.971178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] use_stderr = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.971282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] use_syslog = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.971367 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.971513 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 08:40:48.971591 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.971773 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.971858 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.972013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.972118 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.972301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.972391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.972543 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.972623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.972780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.972864 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.973022 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.973099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.973269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.973351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.973506 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.973590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.973748 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.973826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.973996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.974073 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.974233 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.974316 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.974471 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.974555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.974718 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.974802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.974954 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.975032 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.975204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.975281 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.975453 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.975531 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.975680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.975754 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.975925 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.976002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.976203 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.976299 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.976453 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.976537 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.976692 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.976776 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.976946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.977032 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.977181 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.977259 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.977414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.977492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.977650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.977736 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.977889 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.977967 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.978132 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.978210 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.978366 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.978454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.978710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.978946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.979025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.979196 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.979274 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.979451 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.979573 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.979693 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.979783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.979916 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.980001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.980174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.980254 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.980417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.980505 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.980655 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.980733 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.980888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.980967 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.981121 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.981199 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.981354 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.981432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.981578 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.981652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.981813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.981892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.982062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.982146 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.982294 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.982374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.982524 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.982601 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.982755 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.982838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.982992 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.983072 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.983222 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.983300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.983462 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.983568 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.983693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.983798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.983927 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.984038 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.984187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.984293 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.984424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.984529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.984657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.984784 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.984893 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.985001 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.985126 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.985206 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.985361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.985479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.985600 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.985687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.985836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.985917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.986066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.986143 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.986316 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.986399 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.986554 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.986644 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.986789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.986917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.987062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.987201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.987328 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.987461 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.987586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.987714 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.987839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.987972 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.988124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.988285 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.988411 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.988534 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.988657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.988780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.988903 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.989034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.989164 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.989299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.989433 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.989559 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.989709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.989840 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.989979 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.990112 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.990238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.990362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.990487 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.990626 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.990754 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.990924 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.991058 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.991185 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.991314 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.991445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.991579 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.991705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.991844 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.992045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.992191 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.992331 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.992459 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.992582 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.992714 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.992840 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.992965 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.993096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.993222 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.993348 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.993474 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.993599 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.993735 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.993936 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.994069 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.994214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.994389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.994540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.994668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.994807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.994945 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.995079 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.995211 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.995336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.995460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.995618 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.995753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.995832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.995909 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.995909 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.996049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.996158 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.996245 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.996334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.996412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.996485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.996555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.996626 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.996704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.996777 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.996852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.external = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.996924 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.996994 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.997064 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.997142 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.997215 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] auth.external = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.997286 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.997386 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.997467 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.password = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.997540 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.997612 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.token = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.997682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.997759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.997832 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] auth.password = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.997903 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.997974 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] auth.token = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.998045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.998116 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.998193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.998319 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.998430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.998570 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.998680 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.998805 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.998897 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.999056 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.999137 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.999307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.999395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.999540 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.999630 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.999779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] credential.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:48.999862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.000026 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.000161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.000302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.000396 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.000555 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.000641 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.000797 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.000878 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.001045 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.001133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.001283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.001363 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.001539 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.001617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.001792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.001879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.002033 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.002112 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.002288 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.002371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.002551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.002637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.002811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.002890 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.003046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.003124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.003288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.003377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.003537 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.003617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.003781 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] federation.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.003859 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.004014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.004118 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.004283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.004362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.004523 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.004601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.004753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.004833 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.005001 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.005081 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.005241 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.005325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.005483 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.005563 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.005721 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.005799 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.005960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.006037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.006202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.006281 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.006450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.006537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.006686 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.006765 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.006919 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.006994 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.007158 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.007238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.007394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.007633 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.007771 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.007898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.008024 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.008171 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.008294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.008422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.008560 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.008640 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.008718 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.008792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.008864 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.009043 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.009133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.009292 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.009372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.009546 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.009624 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.010291 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.010387 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.010554 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.010802 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.010890 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.011039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.011147 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.011296 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.011384 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.011545 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.011625 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.011820 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.011900 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.012062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.012194 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.012325 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.012406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.012560 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.012640 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.012800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.012882 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.013031 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.013111 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.013269 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.013349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.013512 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.013600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.013754 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.013834 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.014005 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.014085 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.014240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.014321 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.014490 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.014575 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.014738 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.014824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.014973 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.015052 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.015214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.015291 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.015448 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.015536 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.015680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.015762 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.015917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.016003 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.016169 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.016344 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.016426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.016592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.016672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.016837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.016917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.017091 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.017172 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.017328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.017406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.017566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.017644 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.017801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.017882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.018038 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.018116 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.018269 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.018348 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.018518 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.018599 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.018793 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.018880 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.018994 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.019072 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.019227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.019307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.019462 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.019540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.019694 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.019773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.019938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.020022 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.020216 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.020357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.020468 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.020595 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.020717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.020847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.020955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.021097 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.021177 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.021414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.021621 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.021704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.021867 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.021947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.022098 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.022177 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.022336 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.022417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.022575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.022654 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.022813 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.022892 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.023047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.023128 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.023292 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.023371 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.023525 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.023604 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.023773 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.023854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.024011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.024106 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.024275 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.024355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.024507 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.024587 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.024743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.024821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.024983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.025061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.025221 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.025302 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.025511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.026158 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.026272 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.026455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.026561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.026688 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.026786 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.026935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.027016 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.027176 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.027265 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.027417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.027521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.027658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.027739 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.027906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.027993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.028166 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.028250 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.028404 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.028482 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.028644 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.028736 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.028887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.028965 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.029122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.029201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.029359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.029450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.029599 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] resource.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.029679 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.029847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.029925 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.030095 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.030182 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.030333 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.030413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.030579 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.030657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.030819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.030905 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.031055 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.031134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.031291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] role.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.031368 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.031516 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.031591 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] role.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.031755 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.031836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.031991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] role.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.032071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.032265 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] role.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.032348 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.032498 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.032578 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.032748 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.032828 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.032997 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.033077 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.033244 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.033324 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.033490 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.033568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.033736 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.033817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.033970 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.034048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.034218 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.034299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.034466 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.034546 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.034696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.034779 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.034944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.035025 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.035192 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.035373 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.035453 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.035614 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.035693 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.035867 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.036078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.036190 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.036340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.036421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.036592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.036671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.036839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.036921 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.037086 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.037165 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.037340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.037427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.037574 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.037654 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.037806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.037885 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.038042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.038130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.038277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.038356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.038517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.038595 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.038751 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.038839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.038985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.039064 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.039236 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.039314 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.039473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.039560 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.039705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.039785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.039941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.040020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.040202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.040296 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.040437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.040517 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.040674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] token.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.040754 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.040913 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.041000 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.041235 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.041363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.041474 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.041611 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.041714 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.041874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.041956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.042126 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.042205 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.042367 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.042447 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.042616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.042698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.042861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.042941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.043109 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.043188 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.043349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.043430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.043579 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.043657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.043812 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.043890 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.044058 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.044159 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.044332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.044411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.connection = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.044563 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.044655 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.044805 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.044893 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.045081 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.045159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.045324 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.045403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.connection = **** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.045561 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.045650 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.045801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.045889 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.046039 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.046119 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.046278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.046357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.046514 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.046600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.046750 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.046828 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.046986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.047064 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.047221 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.047307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.047472 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.047551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.047707 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.047785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.047952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.048039 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.048209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.048291 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.048452 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.048532 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.048688 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.048778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.048853 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 08:40:49.048924 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.049156 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.049398 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.049634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 08:40:49.049806 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81476) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 08:40:49.079136 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.079370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.079598 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.079969 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.080235 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.080468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.088890 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.089765 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.117666 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.117964 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.118240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.118477 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.173792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.174606 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.224764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.224993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.225783 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.226026 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.385436 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.385741 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.385887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.386030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.386167 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.386301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.386433 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.386573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.386721 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.386862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.387037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.387037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.410602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.410795 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.410932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.411013 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.411086 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.411200 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.411284 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.411420 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.411503 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.411637 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.411720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.411848 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.411931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.412059 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.423813 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.423969 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.437415 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.437586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.437655 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.437729 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.437851 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.437942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.438070 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.438154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.438284 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.438365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.438496 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.438587 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.438709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81476) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.438791 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 08:40:49.455419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.455634 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 08:40:49.455883 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.455994 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.456077 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 08:40:49.456236 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 08:40:49.456450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.456546 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.456657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 08:40:49.456773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.456934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.457196 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.457275 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.457547 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.457713 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.457980 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.467675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.467984 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.468274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.468360 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.468567 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.468651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.468891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.469178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.475636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.476235 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.478748 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.479183 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.479322 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.479428 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.479613 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.479696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.480290 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.480392 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.480485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.480562 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.480635 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.480707 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.480779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.480852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.481074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.481285 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.481369 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.481442 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.481650 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.481742 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.481932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.482207 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.505001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.505811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.506485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.506778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.507295 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.507585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.512267 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.512558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.512848 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.512931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.513153 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.513444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.519235 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.519531 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.519823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.520133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.523667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.523970 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.524264 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.524408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.524537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.524663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.524825 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.525003 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.525147 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.525278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.525421 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.525545 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.525702 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.525798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.525984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.526256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.543267 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.543561 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.543801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.544116 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.547654 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.547993 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.551002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.551305 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.551561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.551787 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.555558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.556031 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.559477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.559923 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.562757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.563067 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.563208 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.563341 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.563485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.563615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.563745 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.563900 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.564032 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.564205 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.564334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.564607 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.580885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.581195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.581346 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.581471 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.581603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.581730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.581856 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.582071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.588765 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.589133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.590294 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.590579 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.590677 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.590941 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.593258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.593554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.593644 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.593857 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.593940 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.594154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.594239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.594440 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.594525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.594736 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.606146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.606260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.606434 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.606519 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.606715 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.606813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.607001 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.607094 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.607284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.607374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.607559 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.607643 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.607839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.607922 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.624004 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.624163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.627102 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.627204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.630016 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.630160 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.631469 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.631606 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.632974 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.633105 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.633262 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.633399 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.633572 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.633699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.640737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.641118 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.642147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.642589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.645217 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.645562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.645725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.645869 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.646033 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.646119 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.646197 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.646445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.654423 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.654719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.655006 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.655088 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.655333 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.655627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.661968 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81476) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.662256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.662544 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.662626 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 08:40:49.662828 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.662914 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.663115 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.663198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.663392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.663483 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.663673 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.663761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.663946 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.664027 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.664254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.664336 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.664530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.664622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.664813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.664899 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.665087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.665216 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.665359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.665449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.665634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81476) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.665731 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.665918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.666279 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 08:40:49.702054 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5600e0e8e9c0 pid: 81476 (default app) Aug 24 08:40:49.703046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5600e0e8e9c0 pid: 81475 (default app) Aug 24 08:40:49.705672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67745665-e03b-4e7a-bf5b-94959b756fdf None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:40:49.705957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67745665-e03b-4e7a-bf5b-94959b756fdf None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:40:49.706236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67745665-e03b-4e7a-bf5b-94959b756fdf None None] PATH_INFO: `/v3` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:40:49.710415 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1/1] 10.222.0.67 () {50 vars in 791 bytes} [Wed Aug 24 08:40:49 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:40:56.472826 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fa147-2e7f-4a02-95c5-d66bc7d77cbf None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:40:56.473458 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fa147-2e7f-4a02-95c5-d66bc7d77cbf None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:40:56.473458 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fa147-2e7f-4a02-95c5-d66bc7d77cbf None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:40:56.477930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1/2] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:40:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:40:56.485761 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f663c4-8371-42a3-b57a-036b7ef09d16 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:40:56.486378 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f663c4-8371-42a3-b57a-036b7ef09d16 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:40:56.486471 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f663c4-8371-42a3-b57a-036b7ef09d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:40:56.562559 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: INFO dbcounter [None req-f6f663c4-8371-42a3-b57a-036b7ef09d16 None None] Registered counter for database keystone Aug 24 08:40:56.583445 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG oslo_db.sqlalchemy.engines [None req-f6f663c4-8371-42a3-b57a-036b7ef09d16 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81476) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 08:40:56.590727 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writer thread running {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 08:40:56.784003 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG passlib.handlers.bcrypt [None req-f6f663c4-8371-42a3-b57a-036b7ef09d16 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81476) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 08:40:56.784450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG passlib.handlers.bcrypt [None req-f6f663c4-8371-42a3-b57a-036b7ef09d16 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81476) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 08:40:56.842791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f6f663c4-8371-42a3-b57a-036b7ef09d16 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:40:56.844491 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f6f663c4-8371-42a3-b57a-036b7ef09d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:40:56.990418 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 2/3] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:40:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 506 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 24 08:40:57.285501 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fa147-2e7f-4a02-95c5-d66bc7d77cbf None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:40:57.355187 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: INFO dbcounter [None req-584fa147-2e7f-4a02-95c5-d66bc7d77cbf None None] Registered counter for database keystone Aug 24 08:40:57.378349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG oslo_db.sqlalchemy.engines [None req-584fa147-2e7f-4a02-95c5-d66bc7d77cbf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81475) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 08:40:57.385496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writer thread running {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 08:40:57.511817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fa147-2e7f-4a02-95c5-d66bc7d77cbf None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:40:57.516361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60131b36-c492-4276-978d-2d6498481487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:40:57.517022 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60131b36-c492-4276-978d-2d6498481487 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:40:57.517308 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60131b36-c492-4276-978d-2d6498481487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:40:57.517583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60131b36-c492-4276-978d-2d6498481487 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:40:57.526610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60131b36-c492-4276-978d-2d6498481487 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:40:57.579935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: WARNING keystone.common.rbac_enforcer.enforcer [None req-60131b36-c492-4276-978d-2d6498481487 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 08:40:57.580212 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60131b36-c492-4276-978d-2d6498481487 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:40:57.590995 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 2/4] 10.222.0.67 () {58 vars in 1324 bytes} [Wed Aug 24 08:40:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:40:57.597127 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f663c4-8371-42a3-b57a-036b7ef09d16 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:40:57.614250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f663c4-8371-42a3-b57a-036b7ef09d16 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:40:57.620084 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e72922a-df1a-4efe-84d0-45e6434b68a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:40:57.621347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72922a-df1a-4efe-84d0-45e6434b68a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:40:57.621629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72922a-df1a-4efe-84d0-45e6434b68a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:40:57.621902 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72922a-df1a-4efe-84d0-45e6434b68a2 admin admin] PATH_INFO: `/v3/projects/456ae4386c984b51b020a86192cad430` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:40:57.623871 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72922a-df1a-4efe-84d0-45e6434b68a2 admin admin] RBAC: Authorizing `identity:get_project(project_id=456ae4386c984b51b020a86192cad430)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:40:57.680774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6e72922a-df1a-4efe-84d0-45e6434b68a2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 08:40:57.681669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72922a-df1a-4efe-84d0-45e6434b68a2 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:40:57.687513 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 3/5] 10.222.0.67 () {56 vars in 1207 bytes} [Wed Aug 24 08:40:57 2022] GET /identity/v3/projects/456ae4386c984b51b020a86192cad430 => generated 344 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:40:59.765405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a00150-e022-4ac1-86b9-00b542b8a2c9 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:40:59.765949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a00150-e022-4ac1-86b9-00b542b8a2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:40:59.765949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a00150-e022-4ac1-86b9-00b542b8a2c9 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:40:59.770263 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 3/6] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:40:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:40:59.780852 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698bbb4-b00a-4632-a716-a2725fdc2c95 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:40:59.780852 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698bbb4-b00a-4632-a716-a2725fdc2c95 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:40:59.780852 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698bbb4-b00a-4632-a716-a2725fdc2c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:40:59.801859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-c698bbb4-b00a-4632-a716-a2725fdc2c95 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:40:59.803635 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c698bbb4-b00a-4632-a716-a2725fdc2c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:40:59.812695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 4/7] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:40:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 08:41:00.114679 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a00150-e022-4ac1-86b9-00b542b8a2c9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:00.129229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a00150-e022-4ac1-86b9-00b542b8a2c9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:00.135074 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01ffca-848f-45e5-8d09-ef941e591758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:00.135746 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01ffca-848f-45e5-8d09-ef941e591758 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:00.136029 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01ffca-848f-45e5-8d09-ef941e591758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:00.136399 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01ffca-848f-45e5-8d09-ef941e591758 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:00.145347 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01ffca-848f-45e5-8d09-ef941e591758 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:00.147510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01ffca-848f-45e5-8d09-ef941e591758 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:00.159703 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 4/8] 10.222.0.67 () {58 vars in 1324 bytes} [Wed Aug 24 08:41:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:41:00.166958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698bbb4-b00a-4632-a716-a2725fdc2c95 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:00.176757 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698bbb4-b00a-4632-a716-a2725fdc2c95 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:00.181578 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb166afe-1286-4ad7-ac46-dd8e9b263625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:00.182656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb166afe-1286-4ad7-ac46-dd8e9b263625 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:00.183144 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb166afe-1286-4ad7-ac46-dd8e9b263625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:00.183632 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb166afe-1286-4ad7-ac46-dd8e9b263625 admin admin] PATH_INFO: `/v3/users/d66e21fb9473433681e9a7a5e248373c` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:00.186133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb166afe-1286-4ad7-ac46-dd8e9b263625 admin admin] RBAC: Authorizing `identity:get_user(user_id=d66e21fb9473433681e9a7a5e248373c)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:00.189566 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb166afe-1286-4ad7-ac46-dd8e9b263625 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:00.190735 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 5/9] 10.222.0.67 () {56 vars in 1198 bytes} [Wed Aug 24 08:41:00 2022] GET /identity/v3/users/d66e21fb9473433681e9a7a5e248373c => generated 247 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:02.275744 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb238cca-2bf6-42e5-852d-c3ee1784ceed None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:02.276367 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb238cca-2bf6-42e5-852d-c3ee1784ceed None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:02.276463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb238cca-2bf6-42e5-852d-c3ee1784ceed None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:02.277750 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 5/10] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:41:02.284163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028fe3a6-f071-4b02-bc3c-0ed01b2c3b7a None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:02.284684 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028fe3a6-f071-4b02-bc3c-0ed01b2c3b7a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:02.285015 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028fe3a6-f071-4b02-bc3c-0ed01b2c3b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:02.305611 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-028fe3a6-f071-4b02-bc3c-0ed01b2c3b7a None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:02.309887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-028fe3a6-f071-4b02-bc3c-0ed01b2c3b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:02.375977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c153aa0-f662-435c-8c0b-1de94082fbf2 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:02.376605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c153aa0-f662-435c-8c0b-1de94082fbf2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:02.377156 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c153aa0-f662-435c-8c0b-1de94082fbf2 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:02.379039 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 6/11] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:02.386783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eea936-01c2-470d-9549-0873e1c2d52a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:02.387278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eea936-01c2-470d-9549-0873e1c2d52a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:02.387766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eea936-01c2-470d-9549-0873e1c2d52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:02.405207 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 6/12] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:02.453542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG passlib.handlers.bcrypt [None req-19eea936-01c2-470d-9549-0873e1c2d52a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81475) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 08:41:02.453968 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG passlib.handlers.bcrypt [None req-19eea936-01c2-470d-9549-0873e1c2d52a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81475) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 08:41:02.501255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-19eea936-01c2-470d-9549-0873e1c2d52a None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:02.502873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-19eea936-01c2-470d-9549-0873e1c2d52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:02.625351 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 7/13] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:02.698638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028fe3a6-f071-4b02-bc3c-0ed01b2c3b7a None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:02.729404 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:02.829732 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028fe3a6-f071-4b02-bc3c-0ed01b2c3b7a None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:02.872157 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64eca8-0833-4a78-b929-be7388c18067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:02.872967 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64eca8-0833-4a78-b929-be7388c18067 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:02.873286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64eca8-0833-4a78-b929-be7388c18067 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:02.873695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64eca8-0833-4a78-b929-be7388c18067 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:02.888008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64eca8-0833-4a78-b929-be7388c18067 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:02.897220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-9f64eca8-0833-4a78-b929-be7388c18067 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:41:02.897220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:02.897220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:02.897220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:02.897220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:02.897220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:02.897220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:02.897220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:02.897220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:02.897220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:02.897220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:02.897220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:02.897220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:02.897220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:02.897220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:02.897220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:02.898839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:02.898839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:02.898839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:02.898839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:02.898839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:02.898839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:02.898839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:02.898839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:02.898839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:02.898839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:02.898839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:02.898839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:02.898839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:02.898839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:02.898839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:02.898839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:02.900857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:02.900857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:02.900857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:02.900857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:02.900857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:02.900857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:02.900857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:02.900857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:02.900857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:02.900857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:02.900857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:41:02.900857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:02.900857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 7/14] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:41:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:02.905868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eea936-01c2-470d-9549-0873e1c2d52a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:02.915179 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64eca8-0833-4a78-b929-be7388c18067 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:03.022672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:03.033174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eea936-01c2-470d-9549-0873e1c2d52a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:03.035865 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:03.048786 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64eca8-0833-4a78-b929-be7388c18067 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:03.091580 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e451692-acfd-4ba2-8224-bc634054d771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:03.092511 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e451692-acfd-4ba2-8224-bc634054d771 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:03.092797 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e451692-acfd-4ba2-8224-bc634054d771 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:03.093066 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e451692-acfd-4ba2-8224-bc634054d771 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:03.093935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e451692-acfd-4ba2-8224-bc634054d771 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:03.094241 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e451692-acfd-4ba2-8224-bc634054d771 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:03.095692 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e451692-acfd-4ba2-8224-bc634054d771 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:03.101615 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954025d3-38de-4c53-bc7d-04e118116028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:03.102297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954025d3-38de-4c53-bc7d-04e118116028 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:03.102840 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954025d3-38de-4c53-bc7d-04e118116028 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:03.103288 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954025d3-38de-4c53-bc7d-04e118116028 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:03.109096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 8/15] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:03.116748 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e451692-acfd-4ba2-8224-bc634054d771 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:03.200679 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954025d3-38de-4c53-bc7d-04e118116028 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:03.202093 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954025d3-38de-4c53-bc7d-04e118116028 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:03.246148 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e451692-acfd-4ba2-8224-bc634054d771 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:03.294270 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2af4a-3cd4-4fa7-858e-fcc676b3bfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:03.295396 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2af4a-3cd4-4fa7-858e-fcc676b3bfb0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:03.295672 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2af4a-3cd4-4fa7-858e-fcc676b3bfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:03.295949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2af4a-3cd4-4fa7-858e-fcc676b3bfb0 None admin] PATH_INFO: `/v3/users/d66e21fb9473433681e9a7a5e248373c` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:03.297519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2af4a-3cd4-4fa7-858e-fcc676b3bfb0 None admin] RBAC: Authorizing `identity:get_user(user_id=d66e21fb9473433681e9a7a5e248373c)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:03.298991 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2af4a-3cd4-4fa7-858e-fcc676b3bfb0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:03.301157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 9/16] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:03 2022] GET /identity/v3/users/d66e21fb9473433681e9a7a5e248373c => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:03.302546 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:03.308437 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2af4a-3cd4-4fa7-858e-fcc676b3bfb0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:03.325192 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 8/17] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:41:03.330763 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954025d3-38de-4c53-bc7d-04e118116028 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:03.363739 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:03.438161 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954025d3-38de-4c53-bc7d-04e118116028 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:03.441452 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2af4a-3cd4-4fa7-858e-fcc676b3bfb0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:03.479001 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df936531-f98f-470f-b62a-015344b37c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:03.479990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df936531-f98f-470f-b62a-015344b37c95 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:03.480519 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df936531-f98f-470f-b62a-015344b37c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:03.480965 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df936531-f98f-470f-b62a-015344b37c95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:03.487143 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92199878-c308-4323-8932-ea2a5dfb22a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:03.487789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92199878-c308-4323-8932-ea2a5dfb22a5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:03.488070 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92199878-c308-4323-8932-ea2a5dfb22a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:03.488365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92199878-c308-4323-8932-ea2a5dfb22a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:03.489590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92199878-c308-4323-8932-ea2a5dfb22a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:03.491094 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92199878-c308-4323-8932-ea2a5dfb22a5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:03.492787 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 10/18] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:41:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:03.494722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df936531-f98f-470f-b62a-015344b37c95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:03.496366 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df936531-f98f-470f-b62a-015344b37c95 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:03.498804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92199878-c308-4323-8932-ea2a5dfb22a5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:03.516297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-df936531-f98f-470f-b62a-015344b37c95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:41:03.516297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:03.516297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:03.516297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:03.516297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:03.516297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:03.516297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:03.516297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:03.516297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:03.516297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:03.516297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:03.516297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:03.516297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:41:03.516297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:41:03.516297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:41:03.516297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:41:03.517989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:03.517989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:03.517989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:03.517989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:03.517989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:03.517989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:03.517989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:03.517989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:03.517989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:03.517989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:03.517989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:03.517989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:03.517989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:03.517989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:03.517989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:03.517989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:03.517989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:41:03.518813 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:41:03.518813 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:41:03.518813 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:03.518813 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 9/19] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:41:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:41:03.525550 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df936531-f98f-470f-b62a-015344b37c95 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:03.576935 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:03.627268 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92199878-c308-4323-8932-ea2a5dfb22a5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:03.629291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:03.656261 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df936531-f98f-470f-b62a-015344b37c95 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:03.665648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e22d0f-a45b-4748-aea9-1a3f109230ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:03.666371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e22d0f-a45b-4748-aea9-1a3f109230ad None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:03.666703 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e22d0f-a45b-4748-aea9-1a3f109230ad None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:03.666979 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e22d0f-a45b-4748-aea9-1a3f109230ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:03.667733 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e22d0f-a45b-4748-aea9-1a3f109230ad None admin] RBAC: Adding query filter params (role.id=b13ffa72344c4856901b1ba003e273fd, scope.domain.id=default, user.id=d66e21fb9473433681e9a7a5e248373c) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:03.668042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e22d0f-a45b-4748-aea9-1a3f109230ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:03.669557 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e22d0f-a45b-4748-aea9-1a3f109230ad None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:03.679716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 11/20] 10.222.0.67 () {56 vars in 1322 bytes} [Wed Aug 24 08:41:03 2022] GET /identity/v3/role_assignments?user.id=d66e21fb9473433681e9a7a5e248373c&scope.domain.id=default&role.id=b13ffa72344c4856901b1ba003e273fd => generated 233 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:03.700453 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d81256-ae20-4a0d-9ee4-b751db9f76a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:03.701106 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d81256-ae20-4a0d-9ee4-b751db9f76a0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:03.701384 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d81256-ae20-4a0d-9ee4-b751db9f76a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:03.701653 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d81256-ae20-4a0d-9ee4-b751db9f76a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:03.702329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d81256-ae20-4a0d-9ee4-b751db9f76a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:03.702629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d81256-ae20-4a0d-9ee4-b751db9f76a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:03.704374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d81256-ae20-4a0d-9ee4-b751db9f76a0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:03.725222 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 10/21] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:06.118755 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a2d04-4dbf-4546-8ef7-4c3c4825001d None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:06.119846 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a2d04-4dbf-4546-8ef7-4c3c4825001d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:06.120398 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a2d04-4dbf-4546-8ef7-4c3c4825001d None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:06.122054 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 12/22] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:41:06.128139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af8800-d7b7-420b-ad17-5e9e10425e70 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:06.129073 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af8800-d7b7-420b-ad17-5e9e10425e70 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:06.129628 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af8800-d7b7-420b-ad17-5e9e10425e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:06.150921 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d2af8800-d7b7-420b-ad17-5e9e10425e70 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:06.152517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d2af8800-d7b7-420b-ad17-5e9e10425e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:06.251391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 11/23] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:06.799097 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a2d04-4dbf-4546-8ef7-4c3c4825001d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:06.840003 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874ef14-4bfa-45b4-aab1-3d5f7dfe7854 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:06.840388 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874ef14-4bfa-45b4-aab1-3d5f7dfe7854 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:06.840931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874ef14-4bfa-45b4-aab1-3d5f7dfe7854 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:06.842680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 12/24] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:06.849903 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7b747-dc57-4405-ac45-c4d0865ca851 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:06.850541 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7b747-dc57-4405-ac45-c4d0865ca851 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:06.851061 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7b747-dc57-4405-ac45-c4d0865ca851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:06.893616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7bc7b747-dc57-4405-ac45-c4d0865ca851 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:06.895371 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7bc7b747-dc57-4405-ac45-c4d0865ca851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:06.936489 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:06.985802 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a2d04-4dbf-4546-8ef7-4c3c4825001d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:07.052335 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 13/25] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:07.055499 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dadf2-4782-41e2-bae7-003e448027aa None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:07.055868 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dadf2-4782-41e2-bae7-003e448027aa None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:07.056177 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dadf2-4782-41e2-bae7-003e448027aa None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:07.057726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 14/26] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:07.061385 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e864fd-4d76-4161-b91c-464448d776c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:07.062217 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e864fd-4d76-4161-b91c-464448d776c1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:07.062639 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e864fd-4d76-4161-b91c-464448d776c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:07.063068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e864fd-4d76-4161-b91c-464448d776c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:07.075015 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3607a21-583c-4c4e-b1d8-358028c29ffa None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:07.075546 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3607a21-583c-4c4e-b1d8-358028c29ffa None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:07.075975 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3607a21-583c-4c4e-b1d8-358028c29ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:07.090802 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e864fd-4d76-4161-b91c-464448d776c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:07.095858 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e3607a21-583c-4c4e-b1d8-358028c29ffa None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:07.097328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e3607a21-583c-4c4e-b1d8-358028c29ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:07.106516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-15e864fd-4d76-4161-b91c-464448d776c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:41:07.106516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:07.106516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:07.106516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:07.106516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:07.106516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:07.106516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:07.106516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:07.106516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:07.106516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:07.106516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:07.106516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:07.106516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:07.106516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:07.106516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:07.106516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:07.107573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:07.107573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:07.107573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:07.107573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:07.107573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:07.107573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:07.107573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:07.107573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:07.107573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:07.107573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:07.107573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:07.107573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:07.107573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:07.107573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:07.107573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:07.107573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:07.108435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:07.108435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:07.108435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:07.108435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:07.108435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:07.108435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:07.108435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:07.108435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:07.108435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:07.108435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:07.108435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:41:07.108435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:41:07.108435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 13/27] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:41:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:07.116189 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b58f2-2c58-4faf-bc67-d6a7e6ee9967 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:07.116483 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b58f2-2c58-4faf-bc67-d6a7e6ee9967 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:07.116762 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b58f2-2c58-4faf-bc67-d6a7e6ee9967 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:07.118717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 14/28] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:07 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:07.119626 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b58f2-2c58-4faf-bc67-d6a7e6ee9967 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:07.175315 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:07.226561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 15/29] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:07.228549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2dcaa-8673-4acf-956f-5047ccf349ed None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:07.228843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2dcaa-8673-4acf-956f-5047ccf349ed None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:07.229200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2dcaa-8673-4acf-956f-5047ccf349ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:07.260216 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-cea2dcaa-8673-4acf-956f-5047ccf349ed None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:07.261426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-cea2dcaa-8673-4acf-956f-5047ccf349ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:07.325982 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b58f2-2c58-4faf-bc67-d6a7e6ee9967 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:07.381553 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6e309-7d4a-4ca9-a60b-a091986848af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:07.382198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6e309-7d4a-4ca9-a60b-a091986848af None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:07.382553 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6e309-7d4a-4ca9-a60b-a091986848af None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:07.382877 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6e309-7d4a-4ca9-a60b-a091986848af None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:07.383779 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6e309-7d4a-4ca9-a60b-a091986848af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:07.384233 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6e309-7d4a-4ca9-a60b-a091986848af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:07.385806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6e309-7d4a-4ca9-a60b-a091986848af None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:07.400879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 15/30] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:07.406611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6e309-7d4a-4ca9-a60b-a091986848af None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:07.460470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 16/31] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:07.462393 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2dcaa-8673-4acf-956f-5047ccf349ed None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:07.483310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:07.600704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:07.608781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6e309-7d4a-4ca9-a60b-a091986848af None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:07.653281 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2dcaa-8673-4acf-956f-5047ccf349ed None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:07.707455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730543c5-4f2d-4e6a-b23c-59b0eb42c747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:07.708573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730543c5-4f2d-4e6a-b23c-59b0eb42c747 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:07.709169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730543c5-4f2d-4e6a-b23c-59b0eb42c747 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:07.709677 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730543c5-4f2d-4e6a-b23c-59b0eb42c747 None admin] PATH_INFO: `/v3/users/d66e21fb9473433681e9a7a5e248373c` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:07.711282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730543c5-4f2d-4e6a-b23c-59b0eb42c747 None admin] RBAC: Authorizing `identity:get_user(user_id=d66e21fb9473433681e9a7a5e248373c)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:07.713880 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730543c5-4f2d-4e6a-b23c-59b0eb42c747 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:07.721553 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 16/32] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:07 2022] GET /identity/v3/users/d66e21fb9473433681e9a7a5e248373c => generated 247 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:07.722925 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730543c5-4f2d-4e6a-b23c-59b0eb42c747 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:07.733012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fa753-5ef2-43ea-8914-bee8c9ffd7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:07.733707 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fa753-5ef2-43ea-8914-bee8c9ffd7fa None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:07.733994 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fa753-5ef2-43ea-8914-bee8c9ffd7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:07.734274 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fa753-5ef2-43ea-8914-bee8c9ffd7fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:07.777068 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298fa753-5ef2-43ea-8914-bee8c9ffd7fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:07.778657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298fa753-5ef2-43ea-8914-bee8c9ffd7fa None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:07.817617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-298fa753-5ef2-43ea-8914-bee8c9ffd7fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:41:07.817617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:07.817617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:07.817617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:07.817617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:07.817617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:07.817617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:07.817617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:07.817617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:07.817617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:07.817617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:07.817617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:07.817617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:41:07.817617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:41:07.817617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:41:07.817617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:41:07.818548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:07.818548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:07.818548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:07.818548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:07.818548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:07.818548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:07.818548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:07.818548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:07.818548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:07.818548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:07.818548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:07.818548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:07.818548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:07.818548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:07.818548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:07.818548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:07.818548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:41:07.819371 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:41:07.819371 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:41:07.819371 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:07.819371 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 17/33] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:41:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:07.819798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fa753-5ef2-43ea-8914-bee8c9ffd7fa None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:07.892009 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730543c5-4f2d-4e6a-b23c-59b0eb42c747 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:07.955520 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1364e-1e12-4a82-be08-426e474fb430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:07.956224 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1364e-1e12-4a82-be08-426e474fb430 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:07.956513 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1364e-1e12-4a82-be08-426e474fb430 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:07.956795 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1364e-1e12-4a82-be08-426e474fb430 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:07.957718 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1364e-1e12-4a82-be08-426e474fb430 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:07.959638 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1364e-1e12-4a82-be08-426e474fb430 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:07.965305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:08.036610 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fa753-5ef2-43ea-8914-bee8c9ffd7fa None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:08.077953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb216bb-3be4-4749-9df8-3ee8c774f271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:08.078605 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb216bb-3be4-4749-9df8-3ee8c774f271 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:08.078883 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb216bb-3be4-4749-9df8-3ee8c774f271 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:08.079148 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb216bb-3be4-4749-9df8-3ee8c774f271 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:08.080374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb216bb-3be4-4749-9df8-3ee8c774f271 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:08.081867 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb216bb-3be4-4749-9df8-3ee8c774f271 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:08.088432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 18/34] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:41:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:08.088432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb216bb-3be4-4749-9df8-3ee8c774f271 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:08.160891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 17/35] 10.222.0.67 () {60 vars in 1131 bytes} [Wed Aug 24 08:41:07 2022] POST /identity/v3/roles => generated 219 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:41:08.161892 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1364e-1e12-4a82-be08-426e474fb430 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:08.198339 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:08.238685 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb216bb-3be4-4749-9df8-3ee8c774f271 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:08.281597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e43c0-8b33-4dfe-b518-5b2b8ee51d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:08.282344 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e43c0-8b33-4dfe-b518-5b2b8ee51d88 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:08.282627 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e43c0-8b33-4dfe-b518-5b2b8ee51d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:08.282899 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e43c0-8b33-4dfe-b518-5b2b8ee51d88 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:08.283595 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e43c0-8b33-4dfe-b518-5b2b8ee51d88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:08.283911 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e43c0-8b33-4dfe-b518-5b2b8ee51d88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:08.285368 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e43c0-8b33-4dfe-b518-5b2b8ee51d88 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:08.287224 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:08.306396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 19/36] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:08.308110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945d360-3cb5-4a48-a57a-7cfe0f036892 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:08.308617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945d360-3cb5-4a48-a57a-7cfe0f036892 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:08.308895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945d360-3cb5-4a48-a57a-7cfe0f036892 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:08.310209 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 20/37] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:08.311487 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945d360-3cb5-4a48-a57a-7cfe0f036892 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:08.334221 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1364e-1e12-4a82-be08-426e474fb430 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:08.389583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0da8e1-ba92-40f7-a62f-f04b23fbc354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:08.390214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0da8e1-ba92-40f7-a62f-f04b23fbc354 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:08.390500 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0da8e1-ba92-40f7-a62f-f04b23fbc354 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:08.390771 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0da8e1-ba92-40f7-a62f-f04b23fbc354 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:08.391519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0da8e1-ba92-40f7-a62f-f04b23fbc354 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:08.394821 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0da8e1-ba92-40f7-a62f-f04b23fbc354 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:08.409800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 18/38] 10.222.0.67 () {60 vars in 1131 bytes} [Wed Aug 24 08:41:08 2022] POST /identity/v3/roles => generated 225 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:41:08.410213 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbbeb4-b1a0-4bcd-a093-3ac383bc4c77 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:08.410500 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbbeb4-b1a0-4bcd-a093-3ac383bc4c77 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:08.410777 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbbeb4-b1a0-4bcd-a093-3ac383bc4c77 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:08.412177 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 19/39] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:08.412862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbbeb4-b1a0-4bcd-a093-3ac383bc4c77 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:08.467773 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945d360-3cb5-4a48-a57a-7cfe0f036892 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:08.508213 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d748b54-458e-42b3-a815-761eac37c5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:08.509506 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d748b54-458e-42b3-a815-761eac37c5a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:08.509848 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d748b54-458e-42b3-a815-761eac37c5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:08.510247 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d748b54-458e-42b3-a815-761eac37c5a7 None admin] PATH_INFO: `/v3/domains/default/users/d66e21fb9473433681e9a7a5e248373c/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:08.512341 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d748b54-458e-42b3-a815-761eac37c5a7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d66e21fb9473433681e9a7a5e248373c, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:08.513830 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d748b54-458e-42b3-a815-761eac37c5a7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:08.517646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:08.548972 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbbeb4-b1a0-4bcd-a093-3ac383bc4c77 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:08.594979 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6dcec-3593-47a1-b1a4-1fb63307df74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:08.595891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6dcec-3593-47a1-b1a4-1fb63307df74 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:08.596408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6dcec-3593-47a1-b1a4-1fb63307df74 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:08.596684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6dcec-3593-47a1-b1a4-1fb63307df74 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:08.597340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6dcec-3593-47a1-b1a4-1fb63307df74 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:08.598792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6dcec-3593-47a1-b1a4-1fb63307df74 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:08.631957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 21/40] 10.222.0.67 () {58 vars in 1361 bytes} [Wed Aug 24 08:41:08 2022] PUT /identity/v3/domains/default/users/d66e21fb9473433681e9a7a5e248373c/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:41:08.634293 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eff5aa-a9f9-4b21-a5e2-653708687392 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:08.634553 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eff5aa-a9f9-4b21-a5e2-653708687392 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:08.634902 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eff5aa-a9f9-4b21-a5e2-653708687392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:08.638971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 20/41] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 24 08:41:08 2022] POST /identity/v3/projects => generated 301 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:41:08.642354 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6007287-289a-4eae-8a1d-a0fb8858eb8b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:08.642354 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6007287-289a-4eae-8a1d-a0fb8858eb8b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:08.642354 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6007287-289a-4eae-8a1d-a0fb8858eb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:08.665960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b6007287-289a-4eae-8a1d-a0fb8858eb8b None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:08.667761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b6007287-289a-4eae-8a1d-a0fb8858eb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:08.770330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-48eff5aa-a9f9-4b21-a5e2-653708687392 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:08.772217 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-48eff5aa-a9f9-4b21-a5e2-653708687392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:08.782421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 21/42] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:08.850655 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:08.884791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 22/43] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:09.074103 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6007287-289a-4eae-8a1d-a0fb8858eb8b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:09.135961 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:09.166993 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eff5aa-a9f9-4b21-a5e2-653708687392 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:09.187756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6007287-289a-4eae-8a1d-a0fb8858eb8b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:09.226021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aebb05-244e-4965-9e22-fe9c3c4569f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:09.226978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aebb05-244e-4965-9e22-fe9c3c4569f9 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:09.228023 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aebb05-244e-4965-9e22-fe9c3c4569f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:09.228522 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aebb05-244e-4965-9e22-fe9c3c4569f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:09.229214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aebb05-244e-4965-9e22-fe9c3c4569f9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:09.230610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aebb05-244e-4965-9e22-fe9c3c4569f9 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:09.246740 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 22/44] 10.222.0.67 () {60 vars in 1131 bytes} [Wed Aug 24 08:41:09 2022] POST /identity/v3/roles => generated 223 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:41:09.275905 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eff5aa-a9f9-4b21-a5e2-653708687392 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:09.315169 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e23a46-fa8b-4941-b8f3-71db3f90427e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:09.315833 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e23a46-fa8b-4941-b8f3-71db3f90427e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:09.316131 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e23a46-fa8b-4941-b8f3-71db3f90427e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:09.316410 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e23a46-fa8b-4941-b8f3-71db3f90427e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:09.317579 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e23a46-fa8b-4941-b8f3-71db3f90427e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:09.319272 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e23a46-fa8b-4941-b8f3-71db3f90427e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:09.324975 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 23/45] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:41:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:09.330468 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aebb05-244e-4965-9e22-fe9c3c4569f9 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:09.450685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aebb05-244e-4965-9e22-fe9c3c4569f9 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:09.470828 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:09.489557 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d4ba9-80f0-450c-bfc2-fc2b90f0edb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:09.490177 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d4ba9-80f0-450c-bfc2-fc2b90f0edb2 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:09.490459 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d4ba9-80f0-450c-bfc2-fc2b90f0edb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:09.490731 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d4ba9-80f0-450c-bfc2-fc2b90f0edb2 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:09.491386 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d4ba9-80f0-450c-bfc2-fc2b90f0edb2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:09.492947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d4ba9-80f0-450c-bfc2-fc2b90f0edb2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:09.520117 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 23/46] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 24 08:41:09 2022] POST /identity/v3/projects => generated 312 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:41:10.659824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e69f6-fab5-4634-b4d9-af931dd85995 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:10.660376 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e69f6-fab5-4634-b4d9-af931dd85995 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:10.660376 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e69f6-fab5-4634-b4d9-af931dd85995 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:10.661750 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 24/47] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:10.667277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96436e3-9450-4a58-a58e-deebc5347b53 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:10.667691 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96436e3-9450-4a58-a58e-deebc5347b53 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:10.667832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96436e3-9450-4a58-a58e-deebc5347b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:10.688582 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d96436e3-9450-4a58-a58e-deebc5347b53 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:10.691356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d96436e3-9450-4a58-a58e-deebc5347b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:10.765169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 24/48] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:11.050440 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e69f6-fab5-4634-b4d9-af931dd85995 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:11.114495 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:11.164375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e69f6-fab5-4634-b4d9-af931dd85995 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:11.199432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9a6f4-86b4-4856-91dd-b9a25e78e396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:11.200074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9a6f4-86b4-4856-91dd-b9a25e78e396 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:11.200382 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9a6f4-86b4-4856-91dd-b9a25e78e396 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:11.200657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9a6f4-86b4-4856-91dd-b9a25e78e396 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:11.210719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9a6f4-86b4-4856-91dd-b9a25e78e396 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:11.215803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-86f9a6f4-86b4-4856-91dd-b9a25e78e396 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:41:11.215803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:11.215803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:11.215803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:11.215803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:11.215803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:11.215803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:11.215803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:11.215803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:11.215803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:11.215803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:11.215803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:11.215803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:11.215803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:11.215803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:11.215803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:11.216696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:11.216696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:11.216696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:11.216696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:11.216696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:11.216696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:11.216696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:11.216696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:11.216696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:11.216696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:11.216696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:11.216696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:11.216696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:11.216696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:11.216696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:11.216696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:11.217444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:11.217444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:11.217444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:11.217444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:11.217444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:11.217444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:11.217444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:11.217444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:11.217444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:11.217444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:11.217444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:41:11.217444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:11.217444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 25/49] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:41:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:11.222718 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96436e3-9450-4a58-a58e-deebc5347b53 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:11.320371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:11.324288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96436e3-9450-4a58-a58e-deebc5347b53 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:11.361632 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6facd-5deb-4b5c-b38e-0aff4d6b4ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:11.362243 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6facd-5deb-4b5c-b38e-0aff4d6b4ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:11.362520 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6facd-5deb-4b5c-b38e-0aff4d6b4ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:11.362790 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6facd-5deb-4b5c-b38e-0aff4d6b4ba1 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:11.363462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6facd-5deb-4b5c-b38e-0aff4d6b4ba1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:11.363761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6facd-5deb-4b5c-b38e-0aff4d6b4ba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:11.365181 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6facd-5deb-4b5c-b38e-0aff4d6b4ba1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:11.374842 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 25/50] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:11.380459 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9a6f4-86b4-4856-91dd-b9a25e78e396 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:11.484082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9a6f4-86b4-4856-91dd-b9a25e78e396 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:11.492536 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108ce4f-5cca-4914-8f6c-1186d88ab1eb None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:11.492902 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108ce4f-5cca-4914-8f6c-1186d88ab1eb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:11.493186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108ce4f-5cca-4914-8f6c-1186d88ab1eb None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:11.494735 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 26/51] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:41:11.497807 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:11.500314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6e332-0c99-4f43-b0fe-6a0817274557 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:11.500595 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6e332-0c99-4f43-b0fe-6a0817274557 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:11.500876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6e332-0c99-4f43-b0fe-6a0817274557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:11.519147 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c1a6e332-0c99-4f43-b0fe-6a0817274557 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:11.520242 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01b153-cc91-43a9-ad98-6fa86937fab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:11.520371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c1a6e332-0c99-4f43-b0fe-6a0817274557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:11.520887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01b153-cc91-43a9-ad98-6fa86937fab4 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:11.521169 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01b153-cc91-43a9-ad98-6fa86937fab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:11.521444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01b153-cc91-43a9-ad98-6fa86937fab4 None admin] PATH_INFO: `/v3/users/d66e21fb9473433681e9a7a5e248373c` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:11.522579 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01b153-cc91-43a9-ad98-6fa86937fab4 None admin] RBAC: Authorizing `identity:get_user(user_id=d66e21fb9473433681e9a7a5e248373c)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:11.524064 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01b153-cc91-43a9-ad98-6fa86937fab4 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:11.526255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 26/52] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:11 2022] GET /identity/v3/users/d66e21fb9473433681e9a7a5e248373c => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:11.531923 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01b153-cc91-43a9-ad98-6fa86937fab4 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:11.591602 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 27/53] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:11.637092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01b153-cc91-43a9-ad98-6fa86937fab4 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:11.671035 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a8892-f66b-484b-aa08-816e2be8e4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:11.671698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a8892-f66b-484b-aa08-816e2be8e4ae None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:11.671943 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a8892-f66b-484b-aa08-816e2be8e4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:11.672257 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a8892-f66b-484b-aa08-816e2be8e4ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:11.673384 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a8892-f66b-484b-aa08-816e2be8e4ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:11.675106 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a8892-f66b-484b-aa08-816e2be8e4ae None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:11.676718 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 27/54] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:41:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:11.682712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6e332-0c99-4f43-b0fe-6a0817274557 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:11.782054 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:11.792088 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6e332-0c99-4f43-b0fe-6a0817274557 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:11.829745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4f7ca-b05e-45c1-91d6-050327b06cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:11.830360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4f7ca-b05e-45c1-91d6-050327b06cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:11.830663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4f7ca-b05e-45c1-91d6-050327b06cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:11.830987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4f7ca-b05e-45c1-91d6-050327b06cf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:11.831724 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4f7ca-b05e-45c1-91d6-050327b06cf1 None admin] RBAC: Adding query filter params (role.id=b13ffa72344c4856901b1ba003e273fd, scope.domain.id=default, user.id=d66e21fb9473433681e9a7a5e248373c) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:11.832033 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4f7ca-b05e-45c1-91d6-050327b06cf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:11.833653 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4f7ca-b05e-45c1-91d6-050327b06cf1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:11.840652 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 28/55] 10.222.0.67 () {56 vars in 1322 bytes} [Wed Aug 24 08:41:11 2022] GET /identity/v3/role_assignments?user.id=d66e21fb9473433681e9a7a5e248373c&scope.domain.id=default&role.id=b13ffa72344c4856901b1ba003e273fd => generated 530 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:11.865373 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a8892-f66b-484b-aa08-816e2be8e4ae None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:11.924687 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:11.980161 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a8892-f66b-484b-aa08-816e2be8e4ae None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:12.015051 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7540d33-0837-4a11-816a-26bb1efffb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:12.015695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7540d33-0837-4a11-816a-26bb1efffb22 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:12.015981 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7540d33-0837-4a11-816a-26bb1efffb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:12.016291 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7540d33-0837-4a11-816a-26bb1efffb22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:12.017406 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7540d33-0837-4a11-816a-26bb1efffb22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:12.018849 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7540d33-0837-4a11-816a-26bb1efffb22 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:12.020493 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 28/56] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:41:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:12.027203 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4f7ca-b05e-45c1-91d6-050327b06cf1 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:12.129302 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4f7ca-b05e-45c1-91d6-050327b06cf1 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:12.162172 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fe49c-71e4-4559-a00d-0265e0b5f3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:12.162784 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fe49c-71e4-4559-a00d-0265e0b5f3d4 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:12.163069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fe49c-71e4-4559-a00d-0265e0b5f3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:12.163346 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fe49c-71e4-4559-a00d-0265e0b5f3d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:12.164003 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7fe49c-71e4-4559-a00d-0265e0b5f3d4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:12.166227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7fe49c-71e4-4559-a00d-0265e0b5f3d4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:12.193687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 29/57] 10.222.0.67 () {60 vars in 1140 bytes} [Wed Aug 24 08:41:12 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:41:14.134702 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7a4d0-7ed4-4366-80a0-fdf206feb746 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:14.135201 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7a4d0-7ed4-4366-80a0-fdf206feb746 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:14.135290 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7a4d0-7ed4-4366-80a0-fdf206feb746 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:14.136663 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 29/58] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:14.141884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876cf90-a96d-4e3f-8282-2db2ff02724d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:14.142277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876cf90-a96d-4e3f-8282-2db2ff02724d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:14.142456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876cf90-a96d-4e3f-8282-2db2ff02724d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:14.160674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8876cf90-a96d-4e3f-8282-2db2ff02724d None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:14.161867 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8876cf90-a96d-4e3f-8282-2db2ff02724d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:14.163861 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:14.229670 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 30/59] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:14.508057 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7a4d0-7ed4-4366-80a0-fdf206feb746 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:14.609966 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7a4d0-7ed4-4366-80a0-fdf206feb746 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:14.627122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:14.646649 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e04a0-c9db-4945-ba16-339b7304edfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:14.647312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e04a0-c9db-4945-ba16-339b7304edfb None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:14.647593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e04a0-c9db-4945-ba16-339b7304edfb None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:14.647864 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e04a0-c9db-4945-ba16-339b7304edfb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:14.649160 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e04a0-c9db-4945-ba16-339b7304edfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:14.651139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e04a0-c9db-4945-ba16-339b7304edfb None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:14.652817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 30/60] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:41:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:14.658398 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876cf90-a96d-4e3f-8282-2db2ff02724d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:14.759179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876cf90-a96d-4e3f-8282-2db2ff02724d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:14.792610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d007471-c157-4ea4-ac2f-8be7fb1b89a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:14.793215 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d007471-c157-4ea4-ac2f-8be7fb1b89a3 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:14.793497 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d007471-c157-4ea4-ac2f-8be7fb1b89a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:14.793769 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d007471-c157-4ea4-ac2f-8be7fb1b89a3 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:14.794431 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d007471-c157-4ea4-ac2f-8be7fb1b89a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:14.795878 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d007471-c157-4ea4-ac2f-8be7fb1b89a3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:14.810917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:14.825316 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 31/61] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 24 08:41:14 2022] POST /identity/v3/users => generated 275 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:41:18.001624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af346f97-1ee1-45c5-b38c-d294f164e32f None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:18.002111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af346f97-1ee1-45c5-b38c-d294f164e32f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:18.002111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af346f97-1ee1-45c5-b38c-d294f164e32f None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:18.003440 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 31/62] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:18.014238 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2ace4-06de-4214-83fa-48a68d35ddc1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:18.015175 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2ace4-06de-4214-83fa-48a68d35ddc1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:18.015653 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2ace4-06de-4214-83fa-48a68d35ddc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:18.036635 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b487955-ece9-4a7b-afa5-ddbc38b05b37 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:18.037252 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b487955-ece9-4a7b-afa5-ddbc38b05b37 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:18.037731 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b487955-ece9-4a7b-afa5-ddbc38b05b37 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:18.038713 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-99f2ace4-06de-4214-83fa-48a68d35ddc1 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:18.040032 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 32/63] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:18.043755 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-99f2ace4-06de-4214-83fa-48a68d35ddc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:18.045733 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df74148-22b7-4ca8-bd6a-7162f031b865 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:18.046020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df74148-22b7-4ca8-bd6a-7162f031b865 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:18.046296 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df74148-22b7-4ca8-bd6a-7162f031b865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:18.075021 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9df74148-22b7-4ca8-bd6a-7162f031b865 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:18.076259 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9df74148-22b7-4ca8-bd6a-7162f031b865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:18.192928 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 32/64] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:18.218157 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 33/65] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:18.330312 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16650a8-5705-4979-a3fa-5579c02f2da1 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:18.330915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16650a8-5705-4979-a3fa-5579c02f2da1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:18.331382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16650a8-5705-4979-a3fa-5579c02f2da1 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:18.332985 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 33/66] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:18.338926 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0f5cd-4442-4685-b8e4-be0f36c75e6e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:18.339215 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0f5cd-4442-4685-b8e4-be0f36c75e6e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:18.339494 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0f5cd-4442-4685-b8e4-be0f36c75e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:18.369175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-c9f0f5cd-4442-4685-b8e4-be0f36c75e6e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:18.370387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c9f0f5cd-4442-4685-b8e4-be0f36c75e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:18.442356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81682c6-7478-44f6-811b-7ba535af1c66 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:18.442976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81682c6-7478-44f6-811b-7ba535af1c66 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:18.443441 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81682c6-7478-44f6-811b-7ba535af1c66 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:18.447092 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 34/67] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:18.452490 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdbafab-dfa7-4258-848b-672a1d63413e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:18.453061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdbafab-dfa7-4258-848b-672a1d63413e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:18.453669 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdbafab-dfa7-4258-848b-672a1d63413e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:18.473260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 34/68] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:18.481734 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3bdbafab-dfa7-4258-848b-672a1d63413e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:18.483239 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3bdbafab-dfa7-4258-848b-672a1d63413e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:18.567024 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 35/69] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:18.593487 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0f5cd-4442-4685-b8e4-be0f36c75e6e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:18.606367 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837eb24-2901-41b1-a8cd-9dffb6014d4b None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:18.606979 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837eb24-2901-41b1-a8cd-9dffb6014d4b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:18.607742 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837eb24-2901-41b1-a8cd-9dffb6014d4b None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:18.611372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 36/70] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:18.616557 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7590a23-185b-4387-b07d-cd43ca187fed None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:18.616842 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7590a23-185b-4387-b07d-cd43ca187fed None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:18.617347 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7590a23-185b-4387-b07d-cd43ca187fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:18.639233 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b7590a23-185b-4387-b07d-cd43ca187fed None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:18.640861 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b7590a23-185b-4387-b07d-cd43ca187fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:18.661840 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:18.721634 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:18.735464 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0f5cd-4442-4685-b8e4-be0f36c75e6e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:18.749954 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 37/71] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:18.750900 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7590a23-185b-4387-b07d-cd43ca187fed None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:18.797543 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82fdfd-b311-489d-a1a7-62dc06e8e140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:18.798438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82fdfd-b311-489d-a1a7-62dc06e8e140 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:18.798855 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82fdfd-b311-489d-a1a7-62dc06e8e140 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:18.799265 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82fdfd-b311-489d-a1a7-62dc06e8e140 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:18.826507 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82fdfd-b311-489d-a1a7-62dc06e8e140 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:18.834624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-dc82fdfd-b311-489d-a1a7-62dc06e8e140 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:41:18.834624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:18.834624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:18.834624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:18.834624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:18.834624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:18.834624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:18.834624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:18.834624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:18.834624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:18.834624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:18.834624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:18.834624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:18.834624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:18.834624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:18.834624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:18.836811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:18.836811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:18.836811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:18.836811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:18.836811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:18.836811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:18.836811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:18.836811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:18.836811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:18.836811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:18.836811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:18.836811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:18.836811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:18.836811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:18.836811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:18.836811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:18.838586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:18.838586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:18.838586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:18.838586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:18.838586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:18.838586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:18.838586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:18.838586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:18.838586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:18.838586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:18.838586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:41:18.838586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:18.841720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 35/72] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:41:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:18.843181 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82fdfd-b311-489d-a1a7-62dc06e8e140 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:18.917002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7590a23-185b-4387-b07d-cd43ca187fed None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:18.987693 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82fdfd-b311-489d-a1a7-62dc06e8e140 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:18.991778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:19.033882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52b0ff-4e81-4275-94fe-09cfcbb44e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:19.034524 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52b0ff-4e81-4275-94fe-09cfcbb44e3e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:19.034810 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52b0ff-4e81-4275-94fe-09cfcbb44e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:19.035087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52b0ff-4e81-4275-94fe-09cfcbb44e3e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:19.047406 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52b0ff-4e81-4275-94fe-09cfcbb44e3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:19.052485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-5e52b0ff-4e81-4275-94fe-09cfcbb44e3e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:41:19.052485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:19.052485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:19.052485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:19.052485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:19.052485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:19.052485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:19.052485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:19.052485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:19.052485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:19.052485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:19.052485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:19.052485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:19.052485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:19.052485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:19.052485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:19.053338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:19.053338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:19.053338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:19.053338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:19.053338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:19.053338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:19.053338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:19.053338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:19.053338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:19.053338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:19.053338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:19.053338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:19.053338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:19.053338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:19.053338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:19.053338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:19.054101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:19.054101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:19.054101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:19.054101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:19.054101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:19.054101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:19.054101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:19.054101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:19.054101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:19.054101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:19.054101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:41:19.054101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:19.054101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 36/73] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:41:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:19.054715 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52b0ff-4e81-4275-94fe-09cfcbb44e3e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:19.073736 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c1f0b-94d6-4c8d-9065-e4119e0d8a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:19.077306 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c1f0b-94d6-4c8d-9065-e4119e0d8a54 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:19.077306 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c1f0b-94d6-4c8d-9065-e4119e0d8a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:19.077306 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c1f0b-94d6-4c8d-9065-e4119e0d8a54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:19.090356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c1f0b-94d6-4c8d-9065-e4119e0d8a54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:19.095584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-9d0c1f0b-94d6-4c8d-9065-e4119e0d8a54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:19.095584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:19.095584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:19.095584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:19.095584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:19.095584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:19.095584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:19.095584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:19.095584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:19.095584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:19.095584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:19.095584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:19.095584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:19.095584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:19.095584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:19.095584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:19.096495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:19.096495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:19.096495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:19.096495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:19.096495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:19.096495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:19.096495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:19.096495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:19.096495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:19.096495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:19.096495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:19.096495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:19.096495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:19.096495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:19.096495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:19.096495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:19.097255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:19.097255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:19.097255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:19.097255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:19.097255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:19.097255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:19.097255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:19.097255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:19.097255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:19.097255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:19.097255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:19.097255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:41:19.097255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 38/74] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:19.097875 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c1f0b-94d6-4c8d-9065-e4119e0d8a54 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:19.155470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:19.175218 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52b0ff-4e81-4275-94fe-09cfcbb44e3e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:19.212677 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d07dd3-beb4-40bc-b1b5-ff3694142ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:19.213309 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d07dd3-beb4-40bc-b1b5-ff3694142ebc None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:19.213594 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d07dd3-beb4-40bc-b1b5-ff3694142ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:19.213868 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d07dd3-beb4-40bc-b1b5-ff3694142ebc None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:19.214550 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d07dd3-beb4-40bc-b1b5-ff3694142ebc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:19.214857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d07dd3-beb4-40bc-b1b5-ff3694142ebc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:19.216313 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d07dd3-beb4-40bc-b1b5-ff3694142ebc None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:19.219148 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c1f0b-94d6-4c8d-9065-e4119e0d8a54 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:19.230232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 37/75] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:19.231111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d07dd3-beb4-40bc-b1b5-ff3694142ebc None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:19.258418 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9e4a9-6f1f-43ae-b17e-e9ba9ece836c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:19.259212 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9e4a9-6f1f-43ae-b17e-e9ba9ece836c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:19.259495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9e4a9-6f1f-43ae-b17e-e9ba9ece836c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:19.259838 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9e4a9-6f1f-43ae-b17e-e9ba9ece836c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:19.259999 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:19.261049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9e4a9-6f1f-43ae-b17e-e9ba9ece836c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:19.262545 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9e4a9-6f1f-43ae-b17e-e9ba9ece836c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:19.264228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 39/76] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:41:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:19.265034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9e4a9-6f1f-43ae-b17e-e9ba9ece836c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:19.353030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d07dd3-beb4-40bc-b1b5-ff3694142ebc None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:19.377416 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9e4a9-6f1f-43ae-b17e-e9ba9ece836c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:19.389741 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a30ae-dca2-40e3-a6df-8aa2f559111c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:19.391210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a30ae-dca2-40e3-a6df-8aa2f559111c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:19.391210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a30ae-dca2-40e3-a6df-8aa2f559111c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:19.391210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a30ae-dca2-40e3-a6df-8aa2f559111c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:19.392264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:19.402638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a30ae-dca2-40e3-a6df-8aa2f559111c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:19.407339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-958a30ae-dca2-40e3-a6df-8aa2f559111c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:19.407339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:19.407339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:19.407339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:19.407339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:19.407339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:19.407339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:19.407339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:19.407339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:19.407339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:19.407339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:19.407339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:19.407339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:19.407339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:19.407339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:19.407339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:19.408260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:19.408260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:19.408260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:19.408260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:19.408260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:19.408260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:19.408260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:19.408260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:19.408260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:19.408260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:19.408260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:19.408260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:19.408260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:19.408260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:19.408260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:19.408260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:19.409042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:19.409042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:19.409042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:19.409042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:19.409042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:19.409042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:19.409042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:19.409042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:19.409042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:19.409042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:19.409042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:19.409042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:19.409042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 38/77] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:19.409664 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a30ae-dca2-40e3-a6df-8aa2f559111c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:19.427755 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd88cd-4de9-4792-8827-56c598dd77ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:19.428316 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd88cd-4de9-4792-8827-56c598dd77ec None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:19.428598 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd88cd-4de9-4792-8827-56c598dd77ec None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:19.428875 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd88cd-4de9-4792-8827-56c598dd77ec None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:19.429553 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd88cd-4de9-4792-8827-56c598dd77ec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:19.429856 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd88cd-4de9-4792-8827-56c598dd77ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:19.431239 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd88cd-4de9-4792-8827-56c598dd77ec None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:19.439949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 40/78] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:41:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:19.440815 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd88cd-4de9-4792-8827-56c598dd77ec None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:19.510680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:19.528580 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a30ae-dca2-40e3-a6df-8aa2f559111c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:19.554658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd88cd-4de9-4792-8827-56c598dd77ec None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:19.569989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d6d0e-fa43-4602-a49e-b35b2dcafdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:19.570648 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d6d0e-fa43-4602-a49e-b35b2dcafdf6 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:19.571450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d6d0e-fa43-4602-a49e-b35b2dcafdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:19.571722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d6d0e-fa43-4602-a49e-b35b2dcafdf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:19.572434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d6d0e-fa43-4602-a49e-b35b2dcafdf6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:19.572785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d6d0e-fa43-4602-a49e-b35b2dcafdf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:19.574330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d6d0e-fa43-4602-a49e-b35b2dcafdf6 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:19.582767 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 39/79] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:19.583578 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d6d0e-fa43-4602-a49e-b35b2dcafdf6 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:19.596271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4463e88-4625-4a62-8854-c6eddb8a88fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:19.596929 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4463e88-4625-4a62-8854-c6eddb8a88fe None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:19.597213 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4463e88-4625-4a62-8854-c6eddb8a88fe None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:19.597487 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4463e88-4625-4a62-8854-c6eddb8a88fe None admin] PATH_INFO: `/v3/users/d66e21fb9473433681e9a7a5e248373c` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:19.598644 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4463e88-4625-4a62-8854-c6eddb8a88fe None admin] RBAC: Authorizing `identity:get_user(user_id=d66e21fb9473433681e9a7a5e248373c)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:19.600148 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4463e88-4625-4a62-8854-c6eddb8a88fe None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:19.601855 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 41/80] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:19 2022] GET /identity/v3/users/d66e21fb9473433681e9a7a5e248373c => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:19.602615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4463e88-4625-4a62-8854-c6eddb8a88fe None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:19.648426 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:19.702127 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d6d0e-fa43-4602-a49e-b35b2dcafdf6 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:19.708198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4463e88-4625-4a62-8854-c6eddb8a88fe None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:19.748751 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c60ba-9897-4ffa-912a-042509025d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:19.749376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c60ba-9897-4ffa-912a-042509025d6d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:19.749660 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c60ba-9897-4ffa-912a-042509025d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:19.749935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c60ba-9897-4ffa-912a-042509025d6d None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:19.750646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c60ba-9897-4ffa-912a-042509025d6d None admin] RBAC: Adding query filter params (name=member) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:19.750951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c60ba-9897-4ffa-912a-042509025d6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:19.752245 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8e748-fba9-4de9-9910-7e062a239b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:19.752363 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c60ba-9897-4ffa-912a-042509025d6d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:19.752877 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8e748-fba9-4de9-9910-7e062a239b24 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:19.753161 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8e748-fba9-4de9-9910-7e062a239b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:19.753456 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8e748-fba9-4de9-9910-7e062a239b24 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:19.754452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a8e748-fba9-4de9-9910-7e062a239b24 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:19.756079 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a8e748-fba9-4de9-9910-7e062a239b24 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:19.761282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 42/81] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:19.762116 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c60ba-9897-4ffa-912a-042509025d6d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:19.784532 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 40/82] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 24 08:41:19 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:41:19.785377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8e748-fba9-4de9-9910-7e062a239b24 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:19.790155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:19.920937 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c60ba-9897-4ffa-912a-042509025d6d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:19.925668 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:19.929037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8e748-fba9-4de9-9910-7e062a239b24 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:19.967828 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51f817-66e3-4379-9cea-e3ef2364f4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:19.968450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51f817-66e3-4379-9cea-e3ef2364f4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:19.968728 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51f817-66e3-4379-9cea-e3ef2364f4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:19.969003 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51f817-66e3-4379-9cea-e3ef2364f4f0 None admin] PATH_INFO: `/v3/users/98c373122e5c4b1a947b0dbd458f5b95` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:19.980691 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2b1fb-5c76-4f2e-aa3c-8aecb1e6979d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:19.981347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2b1fb-5c76-4f2e-aa3c-8aecb1e6979d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:19.981628 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2b1fb-5c76-4f2e-aa3c-8aecb1e6979d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:19.981902 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2b1fb-5c76-4f2e-aa3c-8aecb1e6979d None admin] PATH_INFO: `/v3/users/98c373122e5c4b1a947b0dbd458f5b95` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:19.986435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51f817-66e3-4379-9cea-e3ef2364f4f0 None admin] RBAC: Authorizing `identity:get_user(user_id=98c373122e5c4b1a947b0dbd458f5b95)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:19.987877 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51f817-66e3-4379-9cea-e3ef2364f4f0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:19.989932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 43/83] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:19 2022] GET /identity/v3/users/98c373122e5c4b1a947b0dbd458f5b95 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:19.990897 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51f817-66e3-4379-9cea-e3ef2364f4f0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:19.999318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf2b1fb-5c76-4f2e-aa3c-8aecb1e6979d None admin] RBAC: Authorizing `identity:get_user(user_id=98c373122e5c4b1a947b0dbd458f5b95)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:20.001328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf2b1fb-5c76-4f2e-aa3c-8aecb1e6979d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:20.003792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 41/84] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:19 2022] GET /identity/v3/users/98c373122e5c4b1a947b0dbd458f5b95 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:20.004640 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2b1fb-5c76-4f2e-aa3c-8aecb1e6979d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:20.095711 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:20.143158 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2b1fb-5c76-4f2e-aa3c-8aecb1e6979d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:20.143546 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51f817-66e3-4379-9cea-e3ef2364f4f0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:20.182171 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335d0e0-48f7-4b29-bbac-e4d0f97524cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:20.182801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335d0e0-48f7-4b29-bbac-e4d0f97524cc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:20.183180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335d0e0-48f7-4b29-bbac-e4d0f97524cc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:20.183451 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335d0e0-48f7-4b29-bbac-e4d0f97524cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:20.183572 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f39d3-c5f3-489f-9cf7-2c9bec02acad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:20.184202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f39d3-c5f3-489f-9cf7-2c9bec02acad None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:20.184485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f39d3-c5f3-489f-9cf7-2c9bec02acad None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:20.184759 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f39d3-c5f3-489f-9cf7-2c9bec02acad None admin] PATH_INFO: `/v3/users/98c373122e5c4b1a947b0dbd458f5b95` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:20.185901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f39d3-c5f3-489f-9cf7-2c9bec02acad None admin] RBAC: Authorizing `identity:get_user(user_id=98c373122e5c4b1a947b0dbd458f5b95)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:20.188775 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f39d3-c5f3-489f-9cf7-2c9bec02acad None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:20.190759 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 42/85] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:20 2022] GET /identity/v3/users/98c373122e5c4b1a947b0dbd458f5b95 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:20.191596 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f39d3-c5f3-489f-9cf7-2c9bec02acad None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:20.218519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:20.262021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335d0e0-48f7-4b29-bbac-e4d0f97524cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:20.263588 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335d0e0-48f7-4b29-bbac-e4d0f97524cc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:20.331745 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f39d3-c5f3-489f-9cf7-2c9bec02acad None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:20.367198 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:20.375688 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed57b21-16d3-402c-8555-928f42dbaade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:20.376522 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed57b21-16d3-402c-8555-928f42dbaade None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:20.376898 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed57b21-16d3-402c-8555-928f42dbaade None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:20.377266 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed57b21-16d3-402c-8555-928f42dbaade None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:20.409269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 44/86] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:41:20.410907 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335d0e0-48f7-4b29-bbac-e4d0f97524cc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:20.472735 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed57b21-16d3-402c-8555-928f42dbaade None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:20.475058 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed57b21-16d3-402c-8555-928f42dbaade None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:20.505442 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:20.560752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335d0e0-48f7-4b29-bbac-e4d0f97524cc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:20.600359 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b0176-6ca4-4464-b1d1-b9808e20ce47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:20.601271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b0176-6ca4-4464-b1d1-b9808e20ce47 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:20.601694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b0176-6ca4-4464-b1d1-b9808e20ce47 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:20.602251 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b0176-6ca4-4464-b1d1-b9808e20ce47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:20.610071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 43/87] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:41:20.610988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed57b21-16d3-402c-8555-928f42dbaade None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:20.642819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:20.679351 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b0176-6ca4-4464-b1d1-b9808e20ce47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:20.681387 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b0176-6ca4-4464-b1d1-b9808e20ce47 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:20.722155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed57b21-16d3-402c-8555-928f42dbaade None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:20.755097 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:20.765364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b8d71-7b8c-40c5-bc99-3e33f03ecbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:20.765990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b8d71-7b8c-40c5-bc99-3e33f03ecbdc None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:20.766272 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b8d71-7b8c-40c5-bc99-3e33f03ecbdc None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:20.766545 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b8d71-7b8c-40c5-bc99-3e33f03ecbdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:20.802778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 45/88] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:41:20.804055 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b0176-6ca4-4464-b1d1-b9808e20ce47 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:20.848672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b8d71-7b8c-40c5-bc99-3e33f03ecbdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:20.850584 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b8d71-7b8c-40c5-bc99-3e33f03ecbdc None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:20.888217 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:20.922065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b0176-6ca4-4464-b1d1-b9808e20ce47 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:20.961635 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de0695-f101-46b8-ae4c-e4dca15eb2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:20.962800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de0695-f101-46b8-ae4c-e4dca15eb2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:20.963281 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de0695-f101-46b8-ae4c-e4dca15eb2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:20.963793 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de0695-f101-46b8-ae4c-e4dca15eb2b5 None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:20.965229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de0695-f101-46b8-ae4c-e4dca15eb2b5 None admin] RBAC: Authorizing `identity:get_project(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:20.966898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de0695-f101-46b8-ae4c-e4dca15eb2b5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:20.968847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 46/89] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:20 2022] GET /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:20.969886 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de0695-f101-46b8-ae4c-e4dca15eb2b5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:20.983236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 44/90] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:41:20.983790 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b8d71-7b8c-40c5-bc99-3e33f03ecbdc None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:21.020628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:21.098378 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b8d71-7b8c-40c5-bc99-3e33f03ecbdc None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:21.104499 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de0695-f101-46b8-ae4c-e4dca15eb2b5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:21.131149 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:21.139582 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a635ea-def1-4409-a0c9-b28b3e78eecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:21.140714 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a635ea-def1-4409-a0c9-b28b3e78eecf None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:21.141177 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a635ea-def1-4409-a0c9-b28b3e78eecf None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:21.141647 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a635ea-def1-4409-a0c9-b28b3e78eecf None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:21.143657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a635ea-def1-4409-a0c9-b28b3e78eecf None admin] RBAC: Authorizing `identity:get_project(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:21.145288 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a635ea-def1-4409-a0c9-b28b3e78eecf None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:21.147422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 45/91] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:20 2022] GET /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:21.148319 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a635ea-def1-4409-a0c9-b28b3e78eecf None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:21.159000 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bef15-04a5-483e-9be9-a463bad246be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:21.160443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bef15-04a5-483e-9be9-a463bad246be None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:21.160941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bef15-04a5-483e-9be9-a463bad246be None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:21.161413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bef15-04a5-483e-9be9-a463bad246be None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:21.162935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23bef15-04a5-483e-9be9-a463bad246be None admin] RBAC: Authorizing `identity:get_project(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:21.165174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23bef15-04a5-483e-9be9-a463bad246be None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:21.168002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 47/92] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:20 2022] GET /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:21.169551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bef15-04a5-483e-9be9-a463bad246be None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:21.298847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a635ea-def1-4409-a0c9-b28b3e78eecf None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:21.326655 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:21.333107 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bef15-04a5-483e-9be9-a463bad246be None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:21.341337 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7fc32d-9068-4d84-8745-ab7313164843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:21.341971 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fc32d-9068-4d84-8745-ab7313164843 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:21.342255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fc32d-9068-4d84-8745-ab7313164843 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:21.342538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fc32d-9068-4d84-8745-ab7313164843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:21.344669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7fc32d-9068-4d84-8745-ab7313164843 None admin] RBAC: Adding query filter params (role.id=b13ffa72344c4856901b1ba003e273fd, scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d, user.id=d66e21fb9473433681e9a7a5e248373c) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:21.344992 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7fc32d-9068-4d84-8745-ab7313164843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:21.346469 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7fc32d-9068-4d84-8745-ab7313164843 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:21.378635 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 46/93] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:41:21 2022] GET /identity/v3/role_assignments?user.id=d66e21fb9473433681e9a7a5e248373c&scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d&role.id=b13ffa72344c4856901b1ba003e273fd => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:21.379631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7fc32d-9068-4d84-8745-ab7313164843 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:21.384807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2cce7-b36a-4969-91d0-5250b36f79ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:21.386245 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2cce7-b36a-4969-91d0-5250b36f79ea None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:21.386789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2cce7-b36a-4969-91d0-5250b36f79ea None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:21.387285 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2cce7-b36a-4969-91d0-5250b36f79ea None admin] PATH_INFO: `/v3/projects/181e5e0aca1b4b26890fff4fb7749311` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:21.388820 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2cce7-b36a-4969-91d0-5250b36f79ea None admin] RBAC: Authorizing `identity:get_project(project_id=181e5e0aca1b4b26890fff4fb7749311)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:21.390774 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2cce7-b36a-4969-91d0-5250b36f79ea None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:21.392923 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 48/94] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:21 2022] GET /identity/v3/projects/181e5e0aca1b4b26890fff4fb7749311 => generated 312 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:21.394266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2cce7-b36a-4969-91d0-5250b36f79ea None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:21.522481 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2cce7-b36a-4969-91d0-5250b36f79ea None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:21.564738 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba8f65-b179-41c7-94a4-264bde84955a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:21.565737 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba8f65-b179-41c7-94a4-264bde84955a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:21.566237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba8f65-b179-41c7-94a4-264bde84955a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:21.566829 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba8f65-b179-41c7-94a4-264bde84955a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:21.567809 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba8f65-b179-41c7-94a4-264bde84955a None admin] RBAC: Adding query filter params (role.id=5915b1954b514151ab17ff5d531036e4, scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d, user.id=98c373122e5c4b1a947b0dbd458f5b95) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:21.568357 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba8f65-b179-41c7-94a4-264bde84955a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:21.570144 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba8f65-b179-41c7-94a4-264bde84955a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:21.578797 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 49/95] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:41:21 2022] GET /identity/v3/role_assignments?user.id=98c373122e5c4b1a947b0dbd458f5b95&scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d&role.id=5915b1954b514151ab17ff5d531036e4 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:21.580293 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba8f65-b179-41c7-94a4-264bde84955a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:21.640500 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:21.666841 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:21.712613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba8f65-b179-41c7-94a4-264bde84955a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:21.801283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7fc32d-9068-4d84-8745-ab7313164843 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:21.804678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ed864-9ce3-43a7-a5c1-ecdad640d0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:21.805889 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ed864-9ce3-43a7-a5c1-ecdad640d0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:21.806738 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ed864-9ce3-43a7-a5c1-ecdad640d0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:21.806738 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ed864-9ce3-43a7-a5c1-ecdad640d0a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:21.807192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ed864-9ce3-43a7-a5c1-ecdad640d0a1 None admin] RBAC: Adding query filter params (role.id=92533f29c1d54fe09591e6f2a906ac04, scope.project.id=181e5e0aca1b4b26890fff4fb7749311, user.id=98c373122e5c4b1a947b0dbd458f5b95) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:21.807507 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ed864-9ce3-43a7-a5c1-ecdad640d0a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:21.809042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ed864-9ce3-43a7-a5c1-ecdad640d0a1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:21.829770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 50/96] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:41:21 2022] GET /identity/v3/role_assignments?user.id=98c373122e5c4b1a947b0dbd458f5b95&scope.project.id=181e5e0aca1b4b26890fff4fb7749311&role.id=92533f29c1d54fe09591e6f2a906ac04 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:21.876130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d6955-ebe2-4a11-b55b-75db2374e915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:21.877903 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d6955-ebe2-4a11-b55b-75db2374e915 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:21.878186 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d6955-ebe2-4a11-b55b-75db2374e915 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:21.878459 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d6955-ebe2-4a11-b55b-75db2374e915 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:21.879240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d6955-ebe2-4a11-b55b-75db2374e915 None admin] RBAC: Adding query filter params (role.id=92533f29c1d54fe09591e6f2a906ac04, scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d, user.id=98c373122e5c4b1a947b0dbd458f5b95) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:21.879553 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d6955-ebe2-4a11-b55b-75db2374e915 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:21.881546 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d6955-ebe2-4a11-b55b-75db2374e915 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:21.892353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 47/97] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:41:21 2022] GET /identity/v3/role_assignments?user.id=98c373122e5c4b1a947b0dbd458f5b95&scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d&role.id=92533f29c1d54fe09591e6f2a906ac04 => generated 259 bytes in 513 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:22.277540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61f06c-ccb9-4dee-95d9-7d1ca67be9fa None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:22.278090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61f06c-ccb9-4dee-95d9-7d1ca67be9fa None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:22.278090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61f06c-ccb9-4dee-95d9-7d1ca67be9fa None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:22.279571 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 51/98] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:22.285135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d128a-17ac-4ce8-9c81-96f3da036808 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:22.285579 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d128a-17ac-4ce8-9c81-96f3da036808 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:22.285763 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d128a-17ac-4ce8-9c81-96f3da036808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:22.308734 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-268d128a-17ac-4ce8-9c81-96f3da036808 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:22.310024 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-268d128a-17ac-4ce8-9c81-96f3da036808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:22.372669 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5234c-ce0c-4fff-9b87-9fda325b67bd None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:22.372990 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5234c-ce0c-4fff-9b87-9fda325b67bd None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:22.373241 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5234c-ce0c-4fff-9b87-9fda325b67bd None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:22.374785 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 52/99] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:41:22.380209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded372b-dc1c-4c85-a148-f53a1ecea4dc None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:22.380499 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded372b-dc1c-4c85-a148-f53a1ecea4dc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:22.380780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded372b-dc1c-4c85-a148-f53a1ecea4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:22.402101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4ded372b-dc1c-4c85-a148-f53a1ecea4dc None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:22.403401 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4ded372b-dc1c-4c85-a148-f53a1ecea4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:22.414509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 48/100] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:22.498431 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 53/101] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:22.855177 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d128a-17ac-4ce8-9c81-96f3da036808 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:22.921452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:22.986156 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d128a-17ac-4ce8-9c81-96f3da036808 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:23.031223 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6167135-1328-4075-9abb-87b9a8f09e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:23.031825 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6167135-1328-4075-9abb-87b9a8f09e16 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:23.032079 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6167135-1328-4075-9abb-87b9a8f09e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:23.032382 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6167135-1328-4075-9abb-87b9a8f09e16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:23.033511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6167135-1328-4075-9abb-87b9a8f09e16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:23.034979 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6167135-1328-4075-9abb-87b9a8f09e16 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:23.036634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 49/102] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:41:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:23.042443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded372b-dc1c-4c85-a148-f53a1ecea4dc None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:23.063879 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6167135-1328-4075-9abb-87b9a8f09e16 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:23.103416 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:23.312176 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded372b-dc1c-4c85-a148-f53a1ecea4dc None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:23.348260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6167135-1328-4075-9abb-87b9a8f09e16 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:23.390929 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc9cba-4506-48b3-82a9-2d2051219a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:23.391568 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc9cba-4506-48b3-82a9-2d2051219a75 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:23.391857 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc9cba-4506-48b3-82a9-2d2051219a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:23.392154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc9cba-4506-48b3-82a9-2d2051219a75 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:23.392842 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc9cba-4506-48b3-82a9-2d2051219a75 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:23.395070 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc9cba-4506-48b3-82a9-2d2051219a75 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:23.414677 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:23.457039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fecd0f-bfd2-4eba-b0a1-f2a3bdc09ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:23.457743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fecd0f-bfd2-4eba-b0a1-f2a3bdc09ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:23.457855 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 54/103] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 24 08:41:23 2022] POST /identity/v3/projects => generated 302 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:41:23.458398 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fecd0f-bfd2-4eba-b0a1-f2a3bdc09ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:23.458684 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fecd0f-bfd2-4eba-b0a1-f2a3bdc09ec1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:23.485390 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fecd0f-bfd2-4eba-b0a1-f2a3bdc09ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:23.490210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-b0fecd0f-bfd2-4eba-b0a1-f2a3bdc09ec1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:41:23.490210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:23.490210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:23.490210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:23.490210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:23.490210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:23.490210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:23.490210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:23.490210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:23.490210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:23.490210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:23.490210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:23.490210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:23.490210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:23.490210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:23.490210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:23.491066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:23.491066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:23.491066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:23.491066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:23.491066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:23.491066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:23.491066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:23.491066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:23.491066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:23.491066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:23.491066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:23.491066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:23.491066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:23.491066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:23.491066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:23.491066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:23.491818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:23.491818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:23.491818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:23.491818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:23.491818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:23.491818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:23.491818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:23.491818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:23.491818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:23.491818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:23.491818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:41:23.491818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:23.491818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 50/104] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 428 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:23.499484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc9cba-4506-48b3-82a9-2d2051219a75 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:23.637771 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:23.662962 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc9cba-4506-48b3-82a9-2d2051219a75 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:23.703354 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45322bd-96c2-4b80-af3b-5864e2aeae79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:23.703985 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45322bd-96c2-4b80-af3b-5864e2aeae79 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:23.704421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45322bd-96c2-4b80-af3b-5864e2aeae79 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:23.704719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45322bd-96c2-4b80-af3b-5864e2aeae79 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:23.705590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45322bd-96c2-4b80-af3b-5864e2aeae79 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:23.705989 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45322bd-96c2-4b80-af3b-5864e2aeae79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:23.707379 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45322bd-96c2-4b80-af3b-5864e2aeae79 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:23.717450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 55/105] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:23.722968 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fecd0f-bfd2-4eba-b0a1-f2a3bdc09ec1 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:23.875430 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fecd0f-bfd2-4eba-b0a1-f2a3bdc09ec1 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:23.912577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face72eb-ce3a-4cb9-b07e-78770cf76171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:23.913565 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face72eb-ce3a-4cb9-b07e-78770cf76171 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:23.914076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face72eb-ce3a-4cb9-b07e-78770cf76171 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:23.915372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face72eb-ce3a-4cb9-b07e-78770cf76171 None admin] PATH_INFO: `/v3/users/fc16f80c0e714a4282235bbecea23ffb` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:23.948049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face72eb-ce3a-4cb9-b07e-78770cf76171 None admin] RBAC: Authorizing `identity:get_user(user_id=fc16f80c0e714a4282235bbecea23ffb)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:23.951285 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face72eb-ce3a-4cb9-b07e-78770cf76171 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:23.955633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 51/106] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:23 2022] GET /identity/v3/users/fc16f80c0e714a4282235bbecea23ffb => generated 289 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:23.965792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45322bd-96c2-4b80-af3b-5864e2aeae79 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:24.108731 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45322bd-96c2-4b80-af3b-5864e2aeae79 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:24.152485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5c712-9ee4-43c8-83ca-2bb957163791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:24.153658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5c712-9ee4-43c8-83ca-2bb957163791 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:24.154328 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5c712-9ee4-43c8-83ca-2bb957163791 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:24.154850 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5c712-9ee4-43c8-83ca-2bb957163791 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:24.164517 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:24.245165 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5c712-9ee4-43c8-83ca-2bb957163791 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:24.246663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5c712-9ee4-43c8-83ca-2bb957163791 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:24.399760 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 56/107] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:41:24.405674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face72eb-ce3a-4cb9-b07e-78770cf76171 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:24.458911 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:24.478204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56882d9b-c891-4732-bf62-ade71b1542ec None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:24.478536 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56882d9b-c891-4732-bf62-ade71b1542ec None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:24.478837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56882d9b-c891-4732-bf62-ade71b1542ec None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:24.480243 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 57/108] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:41:24.492987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1792693-f56a-4a6c-82b9-b4dca6c5db5b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:24.493313 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1792693-f56a-4a6c-82b9-b4dca6c5db5b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:24.493555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1792693-f56a-4a6c-82b9-b4dca6c5db5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:24.520949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d1792693-f56a-4a6c-82b9-b4dca6c5db5b None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:24.522656 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d1792693-f56a-4a6c-82b9-b4dca6c5db5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:24.562652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face72eb-ce3a-4cb9-b07e-78770cf76171 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:24.579710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:24.614125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c82366-b419-4ee6-a1ae-43cc9efa33e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:24.614804 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c82366-b419-4ee6-a1ae-43cc9efa33e5 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:24.615089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c82366-b419-4ee6-a1ae-43cc9efa33e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:24.615366 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c82366-b419-4ee6-a1ae-43cc9efa33e5 None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:24.616608 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c82366-b419-4ee6-a1ae-43cc9efa33e5 None admin] RBAC: Authorizing `identity:get_project(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:24.618857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c82366-b419-4ee6-a1ae-43cc9efa33e5 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:24.620726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 52/109] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:24 2022] GET /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:24.627499 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c82366-b419-4ee6-a1ae-43cc9efa33e5 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:24.709905 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 58/110] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:24.831970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c82366-b419-4ee6-a1ae-43cc9efa33e5 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:24.854589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:24.894097 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dbcd2a-a294-4dc1-9728-79d73869a02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:24.895135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbcd2a-a294-4dc1-9728-79d73869a02b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:24.895597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbcd2a-a294-4dc1-9728-79d73869a02b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:24.896054 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbcd2a-a294-4dc1-9728-79d73869a02b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:24.897474 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dbcd2a-a294-4dc1-9728-79d73869a02b None admin] RBAC: Adding query filter params (role.id=7f4cf2414d134bd4ac30986436be0095, scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d, user.id=fc16f80c0e714a4282235bbecea23ffb) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:24.898012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dbcd2a-a294-4dc1-9728-79d73869a02b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:24.901553 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dbcd2a-a294-4dc1-9728-79d73869a02b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:24.913820 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 53/111] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:41:24 2022] GET /identity/v3/role_assignments?user.id=fc16f80c0e714a4282235bbecea23ffb&scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d&role.id=7f4cf2414d134bd4ac30986436be0095 => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:24.920048 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323db56-55a8-41e9-9a0d-4f72f4f09337 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:24.920467 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323db56-55a8-41e9-9a0d-4f72f4f09337 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:24.920822 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323db56-55a8-41e9-9a0d-4f72f4f09337 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:24.922622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 59/112] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:24.931907 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4140cc0-62fb-448a-ac04-a480dd5cdecc None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:24.932561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4140cc0-62fb-448a-ac04-a480dd5cdecc None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:24.933016 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4140cc0-62fb-448a-ac04-a480dd5cdecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:24.957335 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b4140cc0-62fb-448a-ac04-a480dd5cdecc None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:24.959112 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b4140cc0-62fb-448a-ac04-a480dd5cdecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:25.013561 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232003f5-e53f-4bf1-abc4-42671579e3d2 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:25.013808 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232003f5-e53f-4bf1-abc4-42671579e3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:25.014108 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232003f5-e53f-4bf1-abc4-42671579e3d2 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:25.015927 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 60/113] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:25.024269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef88739-3c79-484c-b35a-0826fef02c9e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:25.024269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef88739-3c79-484c-b35a-0826fef02c9e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:25.024269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef88739-3c79-484c-b35a-0826fef02c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:25.042806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2ef88739-3c79-484c-b35a-0826fef02c9e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:25.044264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2ef88739-3c79-484c-b35a-0826fef02c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:25.104880 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 54/114] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:25.160487 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 61/115] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:25.201196 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4140cc0-62fb-448a-ac04-a480dd5cdecc None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:25.326575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4140cc0-62fb-448a-ac04-a480dd5cdecc None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:25.330686 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:25.373929 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6fc30-3051-4a1b-b853-cfc10b9e85bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:25.375395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6fc30-3051-4a1b-b853-cfc10b9e85bb None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:25.375890 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6fc30-3051-4a1b-b853-cfc10b9e85bb None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:25.376473 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6fc30-3051-4a1b-b853-cfc10b9e85bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:25.392423 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb6fc30-3051-4a1b-b853-cfc10b9e85bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:25.397774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-5bb6fc30-3051-4a1b-b853-cfc10b9e85bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:41:25.397774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:25.397774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:25.397774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:25.397774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:25.397774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:25.397774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:25.397774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:25.397774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:25.397774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:25.397774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:25.397774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:25.397774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:25.397774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:25.397774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:25.397774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:25.399355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:25.399355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:25.399355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:25.399355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:25.399355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:25.399355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:25.399355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:25.399355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:25.399355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:25.399355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:25.399355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:25.399355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:25.399355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:25.399355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:25.399355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:25.399355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:25.400361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:25.400361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:25.400361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:25.400361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:25.400361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:25.400361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:25.400361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:25.400361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:25.400361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:25.400361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:25.400361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:41:25.400361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:25.402750 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 55/116] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:41:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:25.409167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef88739-3c79-484c-b35a-0826fef02c9e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:25.518253 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:25.522136 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef88739-3c79-484c-b35a-0826fef02c9e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:25.578479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b16643-9a66-4314-bbf7-4a9da0588599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:25.579528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b16643-9a66-4314-bbf7-4a9da0588599 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:25.579825 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6fc30-3051-4a1b-b853-cfc10b9e85bb None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:25.580318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b16643-9a66-4314-bbf7-4a9da0588599 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:25.580556 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b16643-9a66-4314-bbf7-4a9da0588599 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:25.581993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b16643-9a66-4314-bbf7-4a9da0588599 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:25.582626 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b16643-9a66-4314-bbf7-4a9da0588599 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:25.585561 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b16643-9a66-4314-bbf7-4a9da0588599 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:25.604067 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 62/117] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:41:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:25.610853 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b16643-9a66-4314-bbf7-4a9da0588599 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:25.758534 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6fc30-3051-4a1b-b853-cfc10b9e85bb None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:25.785210 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b16643-9a66-4314-bbf7-4a9da0588599 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:25.795777 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a206041-f935-4e26-8f2a-3dfbd8b62507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:25.796835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a206041-f935-4e26-8f2a-3dfbd8b62507 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:25.797149 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a206041-f935-4e26-8f2a-3dfbd8b62507 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:25.797424 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a206041-f935-4e26-8f2a-3dfbd8b62507 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:25.811272 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a206041-f935-4e26-8f2a-3dfbd8b62507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:25.818000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-1a206041-f935-4e26-8f2a-3dfbd8b62507 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:41:25.818000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:25.818000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:25.818000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:25.818000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:25.818000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:25.818000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:25.818000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:25.818000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:25.818000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:25.818000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:25.818000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:25.818000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:25.818000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:25.818000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:25.818000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:25.818826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:25.818826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:25.818826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:25.818826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:25.818826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:25.818826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:25.818826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:25.818826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:25.818826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:25.818826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:25.818826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:25.818826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:25.818826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:25.818826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:25.818826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:25.818826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:25.819984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:25.819984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:25.819984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:25.819984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:25.819984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:25.819984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:25.819984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:25.819984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:25.819984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:25.819984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:25.819984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:41:25.819984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:25.819984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 56/118] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:41:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:25.821383 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a206041-f935-4e26-8f2a-3dfbd8b62507 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:25.831197 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4481e-8eb8-4a9c-a8a6-162e923f6be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:25.831775 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4481e-8eb8-4a9c-a8a6-162e923f6be0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:25.832059 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4481e-8eb8-4a9c-a8a6-162e923f6be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:25.832378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4481e-8eb8-4a9c-a8a6-162e923f6be0 None admin] PATH_INFO: `/v3/users/98c373122e5c4b1a947b0dbd458f5b95` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:25.833659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4481e-8eb8-4a9c-a8a6-162e923f6be0 None admin] RBAC: Authorizing `identity:get_user(user_id=98c373122e5c4b1a947b0dbd458f5b95)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:25.835267 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4481e-8eb8-4a9c-a8a6-162e923f6be0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:25.837064 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 63/119] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:25 2022] GET /identity/v3/users/98c373122e5c4b1a947b0dbd458f5b95 => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:25.838450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c0021-b6d5-4109-b62b-4050aa5ebf3d None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:25.838742 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c0021-b6d5-4109-b62b-4050aa5ebf3d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:25.839018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c0021-b6d5-4109-b62b-4050aa5ebf3d None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:25.840302 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 64/120] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:25.841009 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c0021-b6d5-4109-b62b-4050aa5ebf3d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:25.855701 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:25.885165 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:25.955042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a206041-f935-4e26-8f2a-3dfbd8b62507 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:25.973009 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c0021-b6d5-4109-b62b-4050aa5ebf3d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:26.008732 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f5b37-28f0-475c-9279-2b55615c8bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:26.009830 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f5b37-28f0-475c-9279-2b55615c8bed None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:26.010124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f5b37-28f0-475c-9279-2b55615c8bed None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:26.010405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f5b37-28f0-475c-9279-2b55615c8bed None admin] PATH_INFO: `/v3/roles/member` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:26.018252 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93c591-dad3-490b-8141-d4e4274b4413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:26.019186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93c591-dad3-490b-8141-d4e4274b4413 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:26.019654 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93c591-dad3-490b-8141-d4e4274b4413 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:26.020143 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93c591-dad3-490b-8141-d4e4274b4413 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:26.021007 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93c591-dad3-490b-8141-d4e4274b4413 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:26.021495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93c591-dad3-490b-8141-d4e4274b4413 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:26.023052 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93c591-dad3-490b-8141-d4e4274b4413 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:26.023884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f5b37-28f0-475c-9279-2b55615c8bed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:26.030305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-dd1f5b37-28f0-475c-9279-2b55615c8bed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:26.030305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:26.030305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:26.030305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:26.030305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:26.030305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:26.030305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:26.030305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:26.030305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:26.030305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:26.030305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:26.030305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:26.030305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:26.030305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:26.030305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:26.030305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:26.031212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:26.031212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:26.031212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:26.031212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:26.031212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:26.031212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:26.031212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:26.031212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:26.031212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:26.031212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:26.031212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:26.031212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:26.031212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:26.031212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:26.031212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:26.031212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:26.032002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:26.032002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:26.032002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:26.032002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:26.032002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:26.032002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:26.032002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:26.032002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:26.032002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:26.032002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:26.032002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:26.032002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:26.032002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 57/121] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:26.036320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f5b37-28f0-475c-9279-2b55615c8bed None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:26.039696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 65/122] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:26.041519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d16fe8-f5ce-4c23-b907-c86f48228f7e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:26.042003 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d16fe8-f5ce-4c23-b907-c86f48228f7e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:26.045537 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d16fe8-f5ce-4c23-b907-c86f48228f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:26.071292 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-93d16fe8-f5ce-4c23-b907-c86f48228f7e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:26.073558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-93d16fe8-f5ce-4c23-b907-c86f48228f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:26.162864 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:26.188230 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f5b37-28f0-475c-9279-2b55615c8bed None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:26.195716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 66/123] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:26.195716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d16fe8-f5ce-4c23-b907-c86f48228f7e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:26.259513 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:26.276268 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e6db6-beab-45e7-8176-78e5489b3913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:26.276998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e6db6-beab-45e7-8176-78e5489b3913 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:26.277294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e6db6-beab-45e7-8176-78e5489b3913 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:26.277614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e6db6-beab-45e7-8176-78e5489b3913 None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:26.278856 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e6db6-beab-45e7-8176-78e5489b3913 None admin] RBAC: Authorizing `identity:get_project(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:26.280464 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e6db6-beab-45e7-8176-78e5489b3913 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:26.282417 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 58/124] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:26 2022] GET /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:26.283687 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e6db6-beab-45e7-8176-78e5489b3913 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:26.402696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d16fe8-f5ce-4c23-b907-c86f48228f7e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:26.441411 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e6db6-beab-45e7-8176-78e5489b3913 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:26.449193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122ebf2-7676-4731-91fa-62c333c290ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:26.449870 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122ebf2-7676-4731-91fa-62c333c290ce None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:26.450086 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122ebf2-7676-4731-91fa-62c333c290ce None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:26.450354 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122ebf2-7676-4731-91fa-62c333c290ce None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:26.451036 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122ebf2-7676-4731-91fa-62c333c290ce None admin] RBAC: Adding query filter params (name=member) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:26.451336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122ebf2-7676-4731-91fa-62c333c290ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:26.452839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122ebf2-7676-4731-91fa-62c333c290ce None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:26.467532 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 67/125] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:26.467532 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122ebf2-7676-4731-91fa-62c333c290ce None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:26.496194 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d006a11-da92-4ab6-8614-96d074d4779d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:26.496874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d006a11-da92-4ab6-8614-96d074d4779d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:26.497155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d006a11-da92-4ab6-8614-96d074d4779d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:26.497504 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d006a11-da92-4ab6-8614-96d074d4779d None admin] PATH_INFO: `/v3/users/d66e21fb9473433681e9a7a5e248373c` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:26.498872 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d006a11-da92-4ab6-8614-96d074d4779d None admin] RBAC: Authorizing `identity:get_user(user_id=d66e21fb9473433681e9a7a5e248373c)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:26.500487 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d006a11-da92-4ab6-8614-96d074d4779d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:26.502252 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 59/126] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:26 2022] GET /identity/v3/users/d66e21fb9473433681e9a7a5e248373c => generated 247 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:26.503432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac4ee6-147d-4610-bf6b-f3ebe857111e None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:26.503765 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac4ee6-147d-4610-bf6b-f3ebe857111e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:26.504123 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac4ee6-147d-4610-bf6b-f3ebe857111e None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:26.505641 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 60/127] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:26.506135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac4ee6-147d-4610-bf6b-f3ebe857111e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:26.541530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:26.586299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:26.610067 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122ebf2-7676-4731-91fa-62c333c290ce None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:26.653319 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980147d1-70f7-4fea-b8e3-41a33f1ba956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:26.654162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980147d1-70f7-4fea-b8e3-41a33f1ba956 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:26.654446 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980147d1-70f7-4fea-b8e3-41a33f1ba956 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:26.654717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980147d1-70f7-4fea-b8e3-41a33f1ba956 None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d/users/98c373122e5c4b1a947b0dbd458f5b95/roles/5915b1954b514151ab17ff5d531036e4` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:26.658191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980147d1-70f7-4fea-b8e3-41a33f1ba956 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d, user_id=98c373122e5c4b1a947b0dbd458f5b95, role_id=5915b1954b514151ab17ff5d531036e4)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:26.659213 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980147d1-70f7-4fea-b8e3-41a33f1ba956 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:26.676578 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac4ee6-147d-4610-bf6b-f3ebe857111e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:26.679221 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 68/128] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:41:26 2022] PUT /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d/users/98c373122e5c4b1a947b0dbd458f5b95/roles/5915b1954b514151ab17ff5d531036e4 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:41:26.681418 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4b3df-c6ab-4676-9a2f-a2f184e253d1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:26.681718 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4b3df-c6ab-4676-9a2f-a2f184e253d1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:26.682042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4b3df-c6ab-4676-9a2f-a2f184e253d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:26.703520 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a5f4b3df-c6ab-4676-9a2f-a2f184e253d1 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:26.704959 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a5f4b3df-c6ab-4676-9a2f-a2f184e253d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:26.756791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eef2f2-ef49-4a35-bc4e-bc22d6c3f313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:26.756791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eef2f2-ef49-4a35-bc4e-bc22d6c3f313 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:26.756791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eef2f2-ef49-4a35-bc4e-bc22d6c3f313 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:26.756791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eef2f2-ef49-4a35-bc4e-bc22d6c3f313 None admin] PATH_INFO: `/v3/users/98c373122e5c4b1a947b0dbd458f5b95` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:26.758081 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eef2f2-ef49-4a35-bc4e-bc22d6c3f313 None admin] RBAC: Authorizing `identity:get_user(user_id=98c373122e5c4b1a947b0dbd458f5b95)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:26.761658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eef2f2-ef49-4a35-bc4e-bc22d6c3f313 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:26.764830 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 61/129] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:26 2022] GET /identity/v3/users/98c373122e5c4b1a947b0dbd458f5b95 => generated 275 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:26.766439 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eef2f2-ef49-4a35-bc4e-bc22d6c3f313 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:26.792915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 69/130] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:26.793714 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4b3df-c6ab-4676-9a2f-a2f184e253d1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:26.895828 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:26.921973 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:26.932741 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4b3df-c6ab-4676-9a2f-a2f184e253d1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:26.947001 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eef2f2-ef49-4a35-bc4e-bc22d6c3f313 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:26.971086 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9023f4-989b-4d53-b5ca-b5465a2ef8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:26.971654 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9023f4-989b-4d53-b5ca-b5465a2ef8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:26.971936 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9023f4-989b-4d53-b5ca-b5465a2ef8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:26.972262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9023f4-989b-4d53-b5ca-b5465a2ef8c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:26.984465 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9023f4-989b-4d53-b5ca-b5465a2ef8c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:26.989342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-1c9023f4-989b-4d53-b5ca-b5465a2ef8c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:26.989342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:26.989342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:26.989342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:26.989342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:26.989342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:26.989342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:26.989342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:26.989342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:26.989342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:26.989342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:26.989342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:26.989342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:26.989342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:26.989342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:26.989342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:26.990930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:26.990930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:26.990930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:26.990930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:26.990930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:26.990930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:26.990930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:26.990930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:26.990930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:26.990930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:26.990930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:26.990930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:26.990930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:26.990930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:26.990930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:26.990930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:26.992910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:26.992910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:26.992910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:26.992910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:26.992910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:26.992910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:26.992910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:26.992910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:26.992910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:26.992910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:26.992910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:26.992910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:41:26.996210 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 70/131] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:26.998122 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9023f4-989b-4d53-b5ca-b5465a2ef8c0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:27.037514 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a7a76-e67f-4652-8a92-6e402b80a01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:27.038190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a7a76-e67f-4652-8a92-6e402b80a01c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:27.038477 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a7a76-e67f-4652-8a92-6e402b80a01c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:27.038754 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a7a76-e67f-4652-8a92-6e402b80a01c None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:27.040446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a7a76-e67f-4652-8a92-6e402b80a01c None admin] RBAC: Authorizing `identity:get_project(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:27.041921 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a7a76-e67f-4652-8a92-6e402b80a01c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:27.043938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 62/132] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:26 2022] GET /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:27.044776 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a7a76-e67f-4652-8a92-6e402b80a01c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:27.156546 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9023f4-989b-4d53-b5ca-b5465a2ef8c0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:27.199995 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:27.207279 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89b75c-d6d4-42b1-aa86-515273239e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:27.208123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89b75c-d6d4-42b1-aa86-515273239e3c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:27.208453 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89b75c-d6d4-42b1-aa86-515273239e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:27.208846 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89b75c-d6d4-42b1-aa86-515273239e3c None admin] PATH_INFO: `/v3/projects/181e5e0aca1b4b26890fff4fb7749311` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:27.210241 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89b75c-d6d4-42b1-aa86-515273239e3c None admin] RBAC: Authorizing `identity:get_project(project_id=181e5e0aca1b4b26890fff4fb7749311)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:27.212740 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89b75c-d6d4-42b1-aa86-515273239e3c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:27.213727 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a7a76-e67f-4652-8a92-6e402b80a01c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:27.215817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 71/133] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:26 2022] GET /identity/v3/projects/181e5e0aca1b4b26890fff4fb7749311 => generated 312 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:27.216830 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89b75c-d6d4-42b1-aa86-515273239e3c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:27.252765 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ca223-6b50-4497-bd12-7fa991e4f52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:27.253307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ca223-6b50-4497-bd12-7fa991e4f52a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:27.253593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ca223-6b50-4497-bd12-7fa991e4f52a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:27.253872 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ca223-6b50-4497-bd12-7fa991e4f52a None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:27.254556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ca223-6b50-4497-bd12-7fa991e4f52a None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:27.254864 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ca223-6b50-4497-bd12-7fa991e4f52a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:27.256296 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ca223-6b50-4497-bd12-7fa991e4f52a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:27.265724 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 63/134] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:27.266589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ca223-6b50-4497-bd12-7fa991e4f52a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:27.287246 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:27.362579 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89b75c-d6d4-42b1-aa86-515273239e3c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:27.384652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ca223-6b50-4497-bd12-7fa991e4f52a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:27.416379 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a37e7-7597-48e4-8fe3-479740673151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:27.417140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a37e7-7597-48e4-8fe3-479740673151 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:27.417425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a37e7-7597-48e4-8fe3-479740673151 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:27.417696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a37e7-7597-48e4-8fe3-479740673151 None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d/users/d66e21fb9473433681e9a7a5e248373c/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:27.419687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a37e7-7597-48e4-8fe3-479740673151 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d, user_id=d66e21fb9473433681e9a7a5e248373c, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:27.421256 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a37e7-7597-48e4-8fe3-479740673151 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:27.421381 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e6320-f85a-42be-a6a2-b56d9c08bc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:27.422025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e6320-f85a-42be-a6a2-b56d9c08bc3d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:27.422315 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e6320-f85a-42be-a6a2-b56d9c08bc3d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:27.422591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e6320-f85a-42be-a6a2-b56d9c08bc3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:27.423719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e6320-f85a-42be-a6a2-b56d9c08bc3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:27.425508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e6320-f85a-42be-a6a2-b56d9c08bc3d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:27.427109 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 64/135] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:41:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:27.427919 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e6320-f85a-42be-a6a2-b56d9c08bc3d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:27.449665 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 72/136] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:41:27 2022] PUT /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d/users/d66e21fb9473433681e9a7a5e248373c/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:41:27.451503 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a37e7-7597-48e4-8fe3-479740673151 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:27.528406 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:27.545639 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e6320-f85a-42be-a6a2-b56d9c08bc3d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:27.552467 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:27.584068 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de28f5-289f-4bfd-99e8-8ead707e4c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:27.584797 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de28f5-289f-4bfd-99e8-8ead707e4c5b None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:27.585085 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de28f5-289f-4bfd-99e8-8ead707e4c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:27.585364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de28f5-289f-4bfd-99e8-8ead707e4c5b None admin] PATH_INFO: `/v3/projects/181e5e0aca1b4b26890fff4fb7749311/users/98c373122e5c4b1a947b0dbd458f5b95/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:27.587911 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de28f5-289f-4bfd-99e8-8ead707e4c5b None admin] RBAC: Authorizing `identity:create_grant(project_id=181e5e0aca1b4b26890fff4fb7749311, user_id=98c373122e5c4b1a947b0dbd458f5b95, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:27.589369 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de28f5-289f-4bfd-99e8-8ead707e4c5b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:27.607952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 65/137] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:41:27 2022] PUT /identity/v3/projects/181e5e0aca1b4b26890fff4fb7749311/users/98c373122e5c4b1a947b0dbd458f5b95/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:41:27.616721 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de28f5-289f-4bfd-99e8-8ead707e4c5b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:27.616831 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a37e7-7597-48e4-8fe3-479740673151 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:27.674488 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a201c15-136d-46d3-a430-9ca5ec0144fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:27.676779 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a201c15-136d-46d3-a430-9ca5ec0144fd None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:27.677115 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a201c15-136d-46d3-a430-9ca5ec0144fd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:27.677397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a201c15-136d-46d3-a430-9ca5ec0144fd None admin] PATH_INFO: `/v3/users/98c373122e5c4b1a947b0dbd458f5b95` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:27.678725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a201c15-136d-46d3-a430-9ca5ec0144fd None admin] RBAC: Authorizing `identity:get_user(user_id=98c373122e5c4b1a947b0dbd458f5b95)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:27.681032 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a201c15-136d-46d3-a430-9ca5ec0144fd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:27.683696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 73/138] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:27 2022] GET /identity/v3/users/98c373122e5c4b1a947b0dbd458f5b95 => generated 275 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:27.685078 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c7a1c-bd71-47e2-a16b-680462e629ad None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:27.685908 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c7a1c-bd71-47e2-a16b-680462e629ad None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:27.686188 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c7a1c-bd71-47e2-a16b-680462e629ad None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:27.688341 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 74/139] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:27.689119 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c7a1c-bd71-47e2-a16b-680462e629ad None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:27.760267 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de28f5-289f-4bfd-99e8-8ead707e4c5b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:27.807588 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e701111-3b07-4ffb-833d-bbd76502760f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:27.808250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e701111-3b07-4ffb-833d-bbd76502760f None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:27.808536 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e701111-3b07-4ffb-833d-bbd76502760f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:27.808806 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e701111-3b07-4ffb-833d-bbd76502760f None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:27.809877 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e701111-3b07-4ffb-833d-bbd76502760f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:27.811801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e701111-3b07-4ffb-833d-bbd76502760f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:27.850300 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 66/140] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 24 08:41:27 2022] POST /identity/v3/users => generated 283 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:41:27.852628 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72cb0f-1a35-4c11-8717-fc86afcded9d None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:27.853258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72cb0f-1a35-4c11-8717-fc86afcded9d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:27.853765 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72cb0f-1a35-4c11-8717-fc86afcded9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:27.855155 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c7a1c-bd71-47e2-a16b-680462e629ad None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:27.862680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:27.883761 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7c72cb0f-1a35-4c11-8717-fc86afcded9d None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:27.885374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7c72cb0f-1a35-4c11-8717-fc86afcded9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:27.921560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:27.940162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da88ca-7d5e-45aa-94ee-2816e308ba2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:27.942642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da88ca-7d5e-45aa-94ee-2816e308ba2e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:27.943099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da88ca-7d5e-45aa-94ee-2816e308ba2e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:27.943532 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da88ca-7d5e-45aa-94ee-2816e308ba2e None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:27.946518 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da88ca-7d5e-45aa-94ee-2816e308ba2e None admin] RBAC: Authorizing `identity:get_project(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:27.950367 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da88ca-7d5e-45aa-94ee-2816e308ba2e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:27.955132 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 75/141] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:27 2022] GET /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:27.962997 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da88ca-7d5e-45aa-94ee-2816e308ba2e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:27.983107 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 67/142] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:28.137903 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da88ca-7d5e-45aa-94ee-2816e308ba2e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:28.185563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82eaf0f-e699-4ab8-a42c-52849a30af49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:28.186646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82eaf0f-e699-4ab8-a42c-52849a30af49 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:28.187138 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82eaf0f-e699-4ab8-a42c-52849a30af49 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:28.187607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82eaf0f-e699-4ab8-a42c-52849a30af49 None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d/users/98c373122e5c4b1a947b0dbd458f5b95/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:28.189820 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82eaf0f-e699-4ab8-a42c-52849a30af49 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d, user_id=98c373122e5c4b1a947b0dbd458f5b95, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:28.191488 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82eaf0f-e699-4ab8-a42c-52849a30af49 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:28.214670 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 76/143] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:41:27 2022] PUT /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d/users/98c373122e5c4b1a947b0dbd458f5b95/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:41:28.451560 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72cb0f-1a35-4c11-8717-fc86afcded9d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:28.566574 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72cb0f-1a35-4c11-8717-fc86afcded9d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:28.603304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:28.610433 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87d4f5-197c-4f01-a298-71c287519158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:28.611512 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87d4f5-197c-4f01-a298-71c287519158 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:28.612021 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87d4f5-197c-4f01-a298-71c287519158 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:28.612517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87d4f5-197c-4f01-a298-71c287519158 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:28.624535 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87d4f5-197c-4f01-a298-71c287519158 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:28.629726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-2a87d4f5-197c-4f01-a298-71c287519158 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:41:28.629726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:28.629726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:28.629726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:28.629726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:28.629726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:28.629726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:28.629726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:28.629726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:28.629726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:28.629726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:28.629726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:28.629726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:28.629726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:28.629726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:28.629726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:28.630592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:28.630592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:28.630592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:28.630592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:28.630592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:28.630592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:28.630592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:28.630592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:28.630592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:28.630592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:28.630592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:28.630592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:28.630592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:28.630592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:28.630592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:28.630592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:28.631353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:28.631353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:28.631353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:28.631353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:28.631353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:28.631353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:28.631353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:28.631353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:28.631353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:28.631353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:28.631353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:41:28.631353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:28.633734 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 68/144] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:28.639773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82eaf0f-e699-4ab8-a42c-52849a30af49 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:28.712147 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:28.780648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82eaf0f-e699-4ab8-a42c-52849a30af49 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:28.823767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843483f2-0626-413c-9ced-b25402195db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:28.824424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843483f2-0626-413c-9ced-b25402195db8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:28.824710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843483f2-0626-413c-9ced-b25402195db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:28.824985 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843483f2-0626-413c-9ced-b25402195db8 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:28.825801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843483f2-0626-413c-9ced-b25402195db8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:28.826108 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843483f2-0626-413c-9ced-b25402195db8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:28.827569 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843483f2-0626-413c-9ced-b25402195db8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:28.837113 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 77/145] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:28.842971 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87d4f5-197c-4f01-a298-71c287519158 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:28.967053 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87d4f5-197c-4f01-a298-71c287519158 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:29.011463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabbe74-8b5e-4f61-9e54-2ea861492886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:29.012500 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabbe74-8b5e-4f61-9e54-2ea861492886 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:29.012988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabbe74-8b5e-4f61-9e54-2ea861492886 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:29.013562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabbe74-8b5e-4f61-9e54-2ea861492886 None admin] PATH_INFO: `/v3/users/fc16f80c0e714a4282235bbecea23ffb` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:29.014998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabbe74-8b5e-4f61-9e54-2ea861492886 None admin] RBAC: Authorizing `identity:get_user(user_id=fc16f80c0e714a4282235bbecea23ffb)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:29.016706 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabbe74-8b5e-4f61-9e54-2ea861492886 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:29.018666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 69/146] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:28 2022] GET /identity/v3/users/fc16f80c0e714a4282235bbecea23ffb => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:29.024604 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843483f2-0626-413c-9ced-b25402195db8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:29.162823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843483f2-0626-413c-9ced-b25402195db8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:29.200921 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:29.210711 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501777e-b392-4641-aca8-f007b5589ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:29.211986 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501777e-b392-4641-aca8-f007b5589ebf None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:29.212307 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501777e-b392-4641-aca8-f007b5589ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:29.212598 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501777e-b392-4641-aca8-f007b5589ebf None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:29.214124 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501777e-b392-4641-aca8-f007b5589ebf None admin] RBAC: Authorizing `identity:get_project(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:29.215779 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501777e-b392-4641-aca8-f007b5589ebf None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:29.219601 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 78/147] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:29 2022] GET /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:29.225437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabbe74-8b5e-4f61-9e54-2ea861492886 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:29.303592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:29.318421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639f0b9-c57b-4886-8eb9-9cbe5693da9c None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:29.318749 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639f0b9-c57b-4886-8eb9-9cbe5693da9c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:29.318969 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639f0b9-c57b-4886-8eb9-9cbe5693da9c None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:29.320727 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 79/148] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:29.326612 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b277ba-2831-448f-b7ac-17255462f4d4 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:29.326887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b277ba-2831-448f-b7ac-17255462f4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:29.327160 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b277ba-2831-448f-b7ac-17255462f4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:29.349570 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-63b277ba-2831-448f-b7ac-17255462f4d4 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:29.350739 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-63b277ba-2831-448f-b7ac-17255462f4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:29.355148 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabbe74-8b5e-4f61-9e54-2ea861492886 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:29.393967 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1914b34-6b5e-416f-8ec6-1a0860eeb917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:29.394997 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1914b34-6b5e-416f-8ec6-1a0860eeb917 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:29.395479 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1914b34-6b5e-416f-8ec6-1a0860eeb917 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:29.395953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1914b34-6b5e-416f-8ec6-1a0860eeb917 None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d/users/fc16f80c0e714a4282235bbecea23ffb/roles/7f4cf2414d134bd4ac30986436be0095` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:29.405020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1914b34-6b5e-416f-8ec6-1a0860eeb917 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d, user_id=fc16f80c0e714a4282235bbecea23ffb, role_id=7f4cf2414d134bd4ac30986436be0095)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:29.407740 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1914b34-6b5e-416f-8ec6-1a0860eeb917 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:29.436357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 70/149] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:41:29 2022] PUT /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d/users/fc16f80c0e714a4282235bbecea23ffb/roles/7f4cf2414d134bd4ac30986436be0095 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:41:29.459520 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 80/150] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:29.984287 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1914b34-6b5e-416f-8ec6-1a0860eeb917 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:30.097882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1914b34-6b5e-416f-8ec6-1a0860eeb917 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:30.127255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:30.143462 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a35235-7095-4b47-8a34-cdf695560255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:30.144120 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a35235-7095-4b47-8a34-cdf695560255 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:30.144415 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a35235-7095-4b47-8a34-cdf695560255 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:30.144689 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a35235-7095-4b47-8a34-cdf695560255 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:30.158904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a35235-7095-4b47-8a34-cdf695560255 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:30.163742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-63a35235-7095-4b47-8a34-cdf695560255 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:41:30.163742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:30.163742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:30.163742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:30.163742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:30.163742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:30.163742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:30.163742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:30.163742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:30.163742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:30.163742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:30.163742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:30.163742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:30.163742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:30.163742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:30.163742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:30.164847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:30.164847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:30.164847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:30.164847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:30.164847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:30.164847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:30.164847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:30.164847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:30.164847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:30.164847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:30.164847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:30.164847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:30.164847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:30.164847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:30.164847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:30.164847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:30.165597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:30.165597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:30.165597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:30.165597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:30.165597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:30.165597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:30.165597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:30.165597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:30.165597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:30.165597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:30.165597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:41:30.165597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:30.165597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 71/151] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:41:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:30.172608 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b277ba-2831-448f-b7ac-17255462f4d4 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:30.329760 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b277ba-2831-448f-b7ac-17255462f4d4 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:30.351205 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:30.376855 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298e726-cc7d-4210-a878-f7a69f2d2621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:30.377511 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298e726-cc7d-4210-a878-f7a69f2d2621 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:30.377896 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298e726-cc7d-4210-a878-f7a69f2d2621 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:30.378175 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298e726-cc7d-4210-a878-f7a69f2d2621 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:30.378863 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298e726-cc7d-4210-a878-f7a69f2d2621 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:30.379162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298e726-cc7d-4210-a878-f7a69f2d2621 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:30.380582 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298e726-cc7d-4210-a878-f7a69f2d2621 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:30.391107 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 81/152] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:41:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:30.396554 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a35235-7095-4b47-8a34-cdf695560255 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:30.485979 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897190e1-b8bf-454c-86ae-5d00fb6210ee None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:30.486299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897190e1-b8bf-454c-86ae-5d00fb6210ee None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:30.486518 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897190e1-b8bf-454c-86ae-5d00fb6210ee None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:30.487905 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 82/153] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:41:30.494231 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ebb64-b19e-43df-92f1-47a461db6045 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:30.494524 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ebb64-b19e-43df-92f1-47a461db6045 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:30.494767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ebb64-b19e-43df-92f1-47a461db6045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:30.520354 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-603ebb64-b19e-43df-92f1-47a461db6045 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:30.521538 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-603ebb64-b19e-43df-92f1-47a461db6045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:30.572302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a35235-7095-4b47-8a34-cdf695560255 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:30.640138 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ffd301-2322-48f0-b56b-49eac2d1c5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:30.640802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffd301-2322-48f0-b56b-49eac2d1c5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:30.641082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffd301-2322-48f0-b56b-49eac2d1c5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:30.641360 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffd301-2322-48f0-b56b-49eac2d1c5b1 None admin] PATH_INFO: `/v3/users/98c373122e5c4b1a947b0dbd458f5b95` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:30.642520 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ffd301-2322-48f0-b56b-49eac2d1c5b1 None admin] RBAC: Authorizing `identity:get_user(user_id=98c373122e5c4b1a947b0dbd458f5b95)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:30.643976 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ffd301-2322-48f0-b56b-49eac2d1c5b1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:30.649160 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 72/154] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:30 2022] GET /identity/v3/users/98c373122e5c4b1a947b0dbd458f5b95 => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:30.651393 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 83/155] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:30.657261 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ffd301-2322-48f0-b56b-49eac2d1c5b1 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:30.737125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:30.810778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ffd301-2322-48f0-b56b-49eac2d1c5b1 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:30.858621 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d8126-8d24-4f67-b542-e529072ba242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:30.859310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d8126-8d24-4f67-b542-e529072ba242 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:30.859593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d8126-8d24-4f67-b542-e529072ba242 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:30.859867 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d8126-8d24-4f67-b542-e529072ba242 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:30.947905 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d8126-8d24-4f67-b542-e529072ba242 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:30.949353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d8126-8d24-4f67-b542-e529072ba242 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:31.012845 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:31.081046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 73/156] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:41:31.087477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ebb64-b19e-43df-92f1-47a461db6045 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:31.252843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ebb64-b19e-43df-92f1-47a461db6045 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:31.275058 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:31.303458 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bee19-9a12-46fd-8079-15fc0d161f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:31.304123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bee19-9a12-46fd-8079-15fc0d161f1c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:31.304471 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bee19-9a12-46fd-8079-15fc0d161f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:31.304742 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bee19-9a12-46fd-8079-15fc0d161f1c None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:31.308860 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bee19-9a12-46fd-8079-15fc0d161f1c None admin] RBAC: Authorizing `identity:get_project(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:31.310452 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bee19-9a12-46fd-8079-15fc0d161f1c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:31.312486 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 84/157] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:31 2022] GET /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:31.318380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d8126-8d24-4f67-b542-e529072ba242 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:31.378271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bee19-9a12-46fd-8079-15fc0d161f1c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:31.505046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d8126-8d24-4f67-b542-e529072ba242 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:31.575947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bee19-9a12-46fd-8079-15fc0d161f1c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:31.582159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595ed79-5fa6-4a85-bed1-dc29391ebc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:31.583275 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595ed79-5fa6-4a85-bed1-dc29391ebc5f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:31.583701 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595ed79-5fa6-4a85-bed1-dc29391ebc5f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:31.584121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595ed79-5fa6-4a85-bed1-dc29391ebc5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:31.585011 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595ed79-5fa6-4a85-bed1-dc29391ebc5f None admin] RBAC: Adding query filter params (role.id=5915b1954b514151ab17ff5d531036e4, scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d, user.id=98c373122e5c4b1a947b0dbd458f5b95) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:31.585458 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595ed79-5fa6-4a85-bed1-dc29391ebc5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:31.587970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595ed79-5fa6-4a85-bed1-dc29391ebc5f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:31.604350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 74/158] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:41:31 2022] GET /identity/v3/role_assignments?user.id=98c373122e5c4b1a947b0dbd458f5b95&scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d&role.id=5915b1954b514151ab17ff5d531036e4 => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:31.611492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12647656-8d93-4d33-abce-a5f94d855fd4 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:31.612227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12647656-8d93-4d33-abce-a5f94d855fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:31.612710 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12647656-8d93-4d33-abce-a5f94d855fd4 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:31.614377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 75/159] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:31.620246 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c8db2-984d-45f0-95f1-1ff49656b242 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:31.620853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c8db2-984d-45f0-95f1-1ff49656b242 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:31.621380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c8db2-984d-45f0-95f1-1ff49656b242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:31.655922 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219af4a0-b673-4a96-856a-c2d6ef980967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:31.656608 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219af4a0-b673-4a96-856a-c2d6ef980967 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:31.656907 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219af4a0-b673-4a96-856a-c2d6ef980967 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:31.657191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219af4a0-b673-4a96-856a-c2d6ef980967 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:31.661326 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-5b5c8db2-984d-45f0-95f1-1ff49656b242 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:31.662905 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5b5c8db2-984d-45f0-95f1-1ff49656b242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:31.675329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:31.677906 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219af4a0-b673-4a96-856a-c2d6ef980967 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:31.687449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-219af4a0-b673-4a96-856a-c2d6ef980967 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:41:31.687449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:31.687449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:31.687449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:31.687449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:31.687449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:31.687449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:31.687449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:31.687449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:31.687449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:31.687449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:31.687449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:31.687449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:31.687449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:31.687449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:31.687449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:31.688473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:31.688473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:31.688473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:31.688473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:31.688473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:31.688473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:31.688473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:31.688473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:31.688473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:31.688473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:31.688473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:31.688473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:31.688473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:31.688473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:31.688473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:31.688473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:31.689237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:31.689237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:31.689237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:31.689237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:31.689237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:31.689237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:31.689237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:31.689237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:31.689237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:31.689237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:31.689237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:41:31.689237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:41:31.689237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 85/160] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:41:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:31.704949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219af4a0-b673-4a96-856a-c2d6ef980967 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:31.768023 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:31.853554 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 76/161] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:31.913079 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee950378-1459-4d60-a68a-bb6936ab5457 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:31.913388 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee950378-1459-4d60-a68a-bb6936ab5457 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:31.913636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee950378-1459-4d60-a68a-bb6936ab5457 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:31.915947 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 77/162] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:41:31.923989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dec3f2-ca3e-4c11-a856-d01090df98af None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:31.924361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dec3f2-ca3e-4c11-a856-d01090df98af None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:31.924837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dec3f2-ca3e-4c11-a856-d01090df98af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:31.932608 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219af4a0-b673-4a96-856a-c2d6ef980967 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:31.959097 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f6dec3f2-ca3e-4c11-a856-d01090df98af None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:31.960354 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f6dec3f2-ca3e-4c11-a856-d01090df98af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:32.003441 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6c2be-3532-4d83-a231-4c40c74409a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:32.004119 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6c2be-3532-4d83-a231-4c40c74409a6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:32.004379 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6c2be-3532-4d83-a231-4c40c74409a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:32.004653 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6c2be-3532-4d83-a231-4c40c74409a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:32.005405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6c2be-3532-4d83-a231-4c40c74409a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:32.005712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6c2be-3532-4d83-a231-4c40c74409a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:32.007113 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6c2be-3532-4d83-a231-4c40c74409a6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:32.026497 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 86/163] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:32.032521 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6c2be-3532-4d83-a231-4c40c74409a6 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:32.124046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 78/164] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:32.233640 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:32.239092 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6c2be-3532-4d83-a231-4c40c74409a6 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:32.286981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af56490-3746-4a28-89a4-a7cdcffc32de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:32.286981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af56490-3746-4a28-89a4-a7cdcffc32de None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:32.286981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af56490-3746-4a28-89a4-a7cdcffc32de None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:32.288374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af56490-3746-4a28-89a4-a7cdcffc32de None admin] PATH_INFO: `/v3/users/d66e21fb9473433681e9a7a5e248373c` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:32.292901 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af56490-3746-4a28-89a4-a7cdcffc32de None admin] RBAC: Authorizing `identity:get_user(user_id=d66e21fb9473433681e9a7a5e248373c)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:32.294490 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af56490-3746-4a28-89a4-a7cdcffc32de None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:32.296238 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 87/165] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:32 2022] GET /identity/v3/users/d66e21fb9473433681e9a7a5e248373c => generated 247 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:32.302663 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dec3f2-ca3e-4c11-a856-d01090df98af None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:32.316009 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af56490-3746-4a28-89a4-a7cdcffc32de None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:32.384458 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:32.415564 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dec3f2-ca3e-4c11-a856-d01090df98af None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:32.450966 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422f7ab-2b49-4532-9269-4a0b1ecd0c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:32.451595 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422f7ab-2b49-4532-9269-4a0b1ecd0c02 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:32.451881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422f7ab-2b49-4532-9269-4a0b1ecd0c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:32.452187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422f7ab-2b49-4532-9269-4a0b1ecd0c02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:32.530841 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422f7ab-2b49-4532-9269-4a0b1ecd0c02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:32.540489 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422f7ab-2b49-4532-9269-4a0b1ecd0c02 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:32.548945 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af56490-3746-4a28-89a4-a7cdcffc32de None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:32.637841 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:32.650497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 79/166] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:41:32.651000 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadd537-eaa9-4347-a3a5-5120ae0c7464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:32.651718 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadd537-eaa9-4347-a3a5-5120ae0c7464 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:32.652006 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadd537-eaa9-4347-a3a5-5120ae0c7464 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:32.652304 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadd537-eaa9-4347-a3a5-5120ae0c7464 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:32.658063 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422f7ab-2b49-4532-9269-4a0b1ecd0c02 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:32.679152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fadd537-eaa9-4347-a3a5-5120ae0c7464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:32.684819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-9fadd537-eaa9-4347-a3a5-5120ae0c7464 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:32.684819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:32.684819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:32.684819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:32.684819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:32.684819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:32.684819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:32.684819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:32.684819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:32.684819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:32.684819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:32.684819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:32.684819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:32.684819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:32.684819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:32.684819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:32.685719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:32.685719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:32.685719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:32.685719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:32.685719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:32.685719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:32.685719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:32.685719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:32.685719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:32.685719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:32.685719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:32.685719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:32.685719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:32.685719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:32.685719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:32.685719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:32.686700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:32.686700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:32.686700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:32.686700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:32.686700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:32.686700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:32.686700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:32.686700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:32.686700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:32.686700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:32.686700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:32.686700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:41:32.686700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 88/167] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:32.693523 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadd537-eaa9-4347-a3a5-5120ae0c7464 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:32.751930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:32.828692 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422f7ab-2b49-4532-9269-4a0b1ecd0c02 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:32.890328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc81bd-f328-425b-b982-34b1253bc49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:32.891009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc81bd-f328-425b-b982-34b1253bc49d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:32.891278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc81bd-f328-425b-b982-34b1253bc49d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:32.891555 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc81bd-f328-425b-b982-34b1253bc49d None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:32.896971 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc81bd-f328-425b-b982-34b1253bc49d None admin] RBAC: Authorizing `identity:get_project(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:32.899052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc81bd-f328-425b-b982-34b1253bc49d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:32.901249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 80/168] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:32 2022] GET /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:32.902097 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc81bd-f328-425b-b982-34b1253bc49d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:32.935338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadd537-eaa9-4347-a3a5-5120ae0c7464 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:32.980591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:32.994571 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73bd34-3e31-4d92-824a-008bbec33890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:32.995204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73bd34-3e31-4d92-824a-008bbec33890 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:32.995483 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73bd34-3e31-4d92-824a-008bbec33890 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:32.995751 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73bd34-3e31-4d92-824a-008bbec33890 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:32.996460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73bd34-3e31-4d92-824a-008bbec33890 None admin] RBAC: Adding query filter params (name=member) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:32.996763 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73bd34-3e31-4d92-824a-008bbec33890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:33.004344 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73bd34-3e31-4d92-824a-008bbec33890 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:33.013819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 89/169] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:33.018298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73bd34-3e31-4d92-824a-008bbec33890 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:33.057762 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc81bd-f328-425b-b982-34b1253bc49d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:33.093559 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15292b-e9b2-41f0-a66f-02de5f8f6dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:33.094118 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15292b-e9b2-41f0-a66f-02de5f8f6dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:33.094398 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15292b-e9b2-41f0-a66f-02de5f8f6dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:33.094679 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15292b-e9b2-41f0-a66f-02de5f8f6dc2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:33.106188 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15292b-e9b2-41f0-a66f-02de5f8f6dc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:33.111114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-da15292b-e9b2-41f0-a66f-02de5f8f6dc2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:33.111114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:33.111114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:33.111114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:33.111114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:33.111114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:33.111114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:33.111114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:33.111114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:33.111114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:33.111114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:33.111114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:33.111114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:33.111114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:33.111114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:33.111114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:33.112279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:33.112279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:33.112279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:33.112279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:33.112279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:33.112279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:33.112279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:33.112279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:33.112279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:33.112279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:33.112279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:33.112279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:33.112279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:33.112279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:33.112279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:33.112279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:33.113078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:33.113078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:33.113078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:33.113078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:33.113078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:33.113078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:33.113078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:33.113078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:33.113078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:33.113078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:33.113078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:33.113078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:33.113078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 81/170] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:33.113799 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15292b-e9b2-41f0-a66f-02de5f8f6dc2 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:33.194901 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:33.212143 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73bd34-3e31-4d92-824a-008bbec33890 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:33.266019 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda3e73-601c-4348-912a-8985019248a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:33.266712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda3e73-601c-4348-912a-8985019248a8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:33.266986 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda3e73-601c-4348-912a-8985019248a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:33.267264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda3e73-601c-4348-912a-8985019248a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:33.268053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda3e73-601c-4348-912a-8985019248a8 None admin] RBAC: Adding query filter params (role.id=b13ffa72344c4856901b1ba003e273fd, scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d, user.id=d66e21fb9473433681e9a7a5e248373c) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:33.268424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda3e73-601c-4348-912a-8985019248a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:33.269907 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda3e73-601c-4348-912a-8985019248a8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:33.283980 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 90/171] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:41:33 2022] GET /identity/v3/role_assignments?user.id=d66e21fb9473433681e9a7a5e248373c&scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d&role.id=b13ffa72344c4856901b1ba003e273fd => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:33.285025 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87250c-f134-4cb3-9c34-fc3c4ad79276 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:33.285319 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87250c-f134-4cb3-9c34-fc3c4ad79276 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:33.285599 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87250c-f134-4cb3-9c34-fc3c4ad79276 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:33.292801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 91/172] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:33.293734 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87250c-f134-4cb3-9c34-fc3c4ad79276 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:33.294240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15292b-e9b2-41f0-a66f-02de5f8f6dc2 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:33.296416 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:33.397189 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32986635-5029-4f0b-8951-d701b836d12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:33.397855 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32986635-5029-4f0b-8951-d701b836d12f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:33.398287 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32986635-5029-4f0b-8951-d701b836d12f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:33.398641 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32986635-5029-4f0b-8951-d701b836d12f None admin] PATH_INFO: `/v3/users/98c373122e5c4b1a947b0dbd458f5b95` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:33.399997 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32986635-5029-4f0b-8951-d701b836d12f None admin] RBAC: Authorizing `identity:get_user(user_id=98c373122e5c4b1a947b0dbd458f5b95)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:33.401542 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32986635-5029-4f0b-8951-d701b836d12f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:33.403523 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 82/173] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:33 2022] GET /identity/v3/users/98c373122e5c4b1a947b0dbd458f5b95 => generated 275 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:33.409413 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f53a8-bfc5-49bb-bcff-e1b78d0441ad None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:33.409804 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f53a8-bfc5-49bb-bcff-e1b78d0441ad None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:33.409957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f53a8-bfc5-49bb-bcff-e1b78d0441ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:33.444740 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-224f53a8-bfc5-49bb-bcff-e1b78d0441ad None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:33.448791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-224f53a8-bfc5-49bb-bcff-e1b78d0441ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:33.503072 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87250c-f134-4cb3-9c34-fc3c4ad79276 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:33.561164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472f9e6-57e0-46d1-a0d4-f1d9224ef0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:33.561801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472f9e6-57e0-46d1-a0d4-f1d9224ef0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:33.562085 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472f9e6-57e0-46d1-a0d4-f1d9224ef0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:33.562360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472f9e6-57e0-46d1-a0d4-f1d9224ef0a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:33.563046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472f9e6-57e0-46d1-a0d4-f1d9224ef0a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:33.563356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472f9e6-57e0-46d1-a0d4-f1d9224ef0a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:33.564875 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472f9e6-57e0-46d1-a0d4-f1d9224ef0a1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:33.576060 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 92/174] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:33.579045 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 83/175] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:33.583676 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472f9e6-57e0-46d1-a0d4-f1d9224ef0a1 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:33.584917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34464f-7e52-4acd-ae26-0dfa27254d97 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:33.585390 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34464f-7e52-4acd-ae26-0dfa27254d97 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:33.585874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34464f-7e52-4acd-ae26-0dfa27254d97 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:33.587643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 84/176] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:33.589055 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3207593-fd02-4fda-a47c-d49433e5e52f None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:33.589566 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3207593-fd02-4fda-a47c-d49433e5e52f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:33.590138 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3207593-fd02-4fda-a47c-d49433e5e52f None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:33.591833 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 85/177] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:33.598114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3207593-fd02-4fda-a47c-d49433e5e52f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:33.613249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:33.721397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472f9e6-57e0-46d1-a0d4-f1d9224ef0a1 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:33.730852 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:33.736024 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3207593-fd02-4fda-a47c-d49433e5e52f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:33.764450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b5591-bd5b-4061-9533-9e43e8f142aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:33.765310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b5591-bd5b-4061-9533-9e43e8f142aa None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:33.765762 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b5591-bd5b-4061-9533-9e43e8f142aa None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:33.766188 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b5591-bd5b-4061-9533-9e43e8f142aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:33.777651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5c7b3-3593-413a-8669-28d2d464c675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:33.778292 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5c7b3-3593-413a-8669-28d2d464c675 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:33.778573 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5c7b3-3593-413a-8669-28d2d464c675 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:33.778851 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5c7b3-3593-413a-8669-28d2d464c675 None admin] PATH_INFO: `/v3/users/98c373122e5c4b1a947b0dbd458f5b95` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:33.779991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5c7b3-3593-413a-8669-28d2d464c675 None admin] RBAC: Authorizing `identity:get_user(user_id=98c373122e5c4b1a947b0dbd458f5b95)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:33.781561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5c7b3-3593-413a-8669-28d2d464c675 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:33.783181 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 86/178] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:33 2022] GET /identity/v3/users/98c373122e5c4b1a947b0dbd458f5b95 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:33.784340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a400b-8c26-4892-ab6a-ccdc170c7816 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:33.784623 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a400b-8c26-4892-ab6a-ccdc170c7816 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:33.784894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a400b-8c26-4892-ab6a-ccdc170c7816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:33.803100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-3c6a400b-8c26-4892-ab6a-ccdc170c7816 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:33.804308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3c6a400b-8c26-4892-ab6a-ccdc170c7816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:33.853283 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b5591-bd5b-4061-9533-9e43e8f142aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:33.855237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b5591-bd5b-4061-9533-9e43e8f142aa None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:33.872710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:33.882895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 87/179] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:33.884307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddaa2d5-3184-4243-9491-8d384d4185fc None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:33.884595 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddaa2d5-3184-4243-9491-8d384d4185fc None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:33.884870 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddaa2d5-3184-4243-9491-8d384d4185fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:33.906306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-3ddaa2d5-3184-4243-9491-8d384d4185fc None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:33.907504 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3ddaa2d5-3184-4243-9491-8d384d4185fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:33.974852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 93/180] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:41:33.977410 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9970b-a3e7-4ac9-989c-c9a0c4385bc3 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:33.977939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9970b-a3e7-4ac9-989c-c9a0c4385bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:33.978406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9970b-a3e7-4ac9-989c-c9a0c4385bc3 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:33.980129 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 94/181] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:33.981061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9970b-a3e7-4ac9-989c-c9a0c4385bc3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:33.994731 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 88/182] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:33.995594 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddaa2d5-3184-4243-9491-8d384d4185fc None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:34.030654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:34.121201 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddaa2d5-3184-4243-9491-8d384d4185fc None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:34.125339 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9970b-a3e7-4ac9-989c-c9a0c4385bc3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:34.141713 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:34.166409 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12821b2-c0bb-4551-946f-207466a1372d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:34.167031 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12821b2-c0bb-4551-946f-207466a1372d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:34.167325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12821b2-c0bb-4551-946f-207466a1372d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:34.167583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12821b2-c0bb-4551-946f-207466a1372d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:34.171184 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d0a9c-f51c-4b8f-bd59-9922893a6a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:34.172686 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d0a9c-f51c-4b8f-bd59-9922893a6a11 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:34.173221 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d0a9c-f51c-4b8f-bd59-9922893a6a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:34.173693 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d0a9c-f51c-4b8f-bd59-9922893a6a11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:34.186781 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d0a9c-f51c-4b8f-bd59-9922893a6a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:34.191756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-980d0a9c-f51c-4b8f-bd59-9922893a6a11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:41:34.191756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:34.191756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:34.191756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:34.191756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:34.191756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:34.191756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:34.191756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:34.191756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:34.191756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:34.191756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:34.191756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:34.191756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:34.191756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:34.191756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:34.191756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:34.192835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:34.192835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:34.192835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:34.192835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:34.192835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:34.192835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:34.192835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:34.192835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:34.192835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:34.192835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:34.192835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:34.192835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:34.192835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:34.192835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:34.192835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:34.192835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:34.193690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:34.193690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:34.193690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:34.193690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:34.193690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:34.193690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:34.193690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:34.193690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:34.193690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:34.193690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:34.193690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:41:34.193690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:34.195664 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 89/183] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:41:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:34.196639 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d0a9c-f51c-4b8f-bd59-9922893a6a11 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:34.245616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12821b2-c0bb-4551-946f-207466a1372d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:34.247033 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12821b2-c0bb-4551-946f-207466a1372d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:34.291194 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:34.308110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d0a9c-f51c-4b8f-bd59-9922893a6a11 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:34.342814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cecdf7-41fd-4f80-b817-6d12feac376d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:34.343449 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cecdf7-41fd-4f80-b817-6d12feac376d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:34.343728 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cecdf7-41fd-4f80-b817-6d12feac376d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:34.343998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cecdf7-41fd-4f80-b817-6d12feac376d None admin] PATH_INFO: `/v3/projects/181e5e0aca1b4b26890fff4fb7749311` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:34.345164 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cecdf7-41fd-4f80-b817-6d12feac376d None admin] RBAC: Authorizing `identity:get_project(project_id=181e5e0aca1b4b26890fff4fb7749311)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:34.346610 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cecdf7-41fd-4f80-b817-6d12feac376d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:34.348324 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 90/184] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:34 2022] GET /identity/v3/projects/181e5e0aca1b4b26890fff4fb7749311 => generated 312 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:34.354469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 95/185] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:41:34.354469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12821b2-c0bb-4551-946f-207466a1372d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:34.355038 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ade96-7d7f-432f-abca-42da5f655f79 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:34.355038 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ade96-7d7f-432f-abca-42da5f655f79 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:34.355038 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ade96-7d7f-432f-abca-42da5f655f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:34.371148 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:34.378076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-197ade96-7d7f-432f-abca-42da5f655f79 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:34.379327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-197ade96-7d7f-432f-abca-42da5f655f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:34.451327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 91/186] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:34.453056 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ade96-7d7f-432f-abca-42da5f655f79 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:34.467912 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12821b2-c0bb-4551-946f-207466a1372d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:34.505091 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0eb41-31aa-43c5-a7e1-ad25382c956b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:34.505646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0eb41-31aa-43c5-a7e1-ad25382c956b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:34.505928 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0eb41-31aa-43c5-a7e1-ad25382c956b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:34.506204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0eb41-31aa-43c5-a7e1-ad25382c956b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:34.507338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0eb41-31aa-43c5-a7e1-ad25382c956b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:34.508902 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0eb41-31aa-43c5-a7e1-ad25382c956b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:34.510515 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 96/187] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:41:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:34.511284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0eb41-31aa-43c5-a7e1-ad25382c956b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:34.562283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:34.582318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ade96-7d7f-432f-abca-42da5f655f79 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:34.611066 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:34.627699 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c033d-86fd-47d7-b143-dae5a29ef6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:34.628673 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c033d-86fd-47d7-b143-dae5a29ef6ed None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:34.629251 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c033d-86fd-47d7-b143-dae5a29ef6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:34.629533 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c033d-86fd-47d7-b143-dae5a29ef6ed None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:34.630211 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925c033d-86fd-47d7-b143-dae5a29ef6ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:34.630515 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925c033d-86fd-47d7-b143-dae5a29ef6ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:34.631894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925c033d-86fd-47d7-b143-dae5a29ef6ed None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:34.632224 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0eb41-31aa-43c5-a7e1-ad25382c956b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:34.641624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 92/188] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:34.642451 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c033d-86fd-47d7-b143-dae5a29ef6ed None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:34.672235 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676d17a-ec25-4a9a-9652-296114c72654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:34.672884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676d17a-ec25-4a9a-9652-296114c72654 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:34.673166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676d17a-ec25-4a9a-9652-296114c72654 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:34.673434 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676d17a-ec25-4a9a-9652-296114c72654 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:34.684753 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676d17a-ec25-4a9a-9652-296114c72654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:34.689593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-5676d17a-ec25-4a9a-9652-296114c72654 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:41:34.689593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:34.689593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:34.689593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:34.689593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:34.689593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:34.689593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:34.689593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:34.689593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:34.689593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:34.689593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:34.689593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:34.689593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:34.689593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:34.689593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:34.689593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:34.690477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:34.690477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:34.690477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:34.690477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:34.690477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:34.690477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:34.690477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:34.690477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:34.690477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:34.690477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:34.690477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:34.690477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:34.690477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:34.690477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:34.690477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:34.690477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:34.691349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:34.691349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:34.691349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:34.691349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:34.691349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:34.691349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:34.691349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:34.691349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:34.691349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:34.691349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:34.691349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:41:34.691349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:41:34.691349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 97/189] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:41:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:34.692168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676d17a-ec25-4a9a-9652-296114c72654 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:34.767391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c033d-86fd-47d7-b143-dae5a29ef6ed None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:34.797804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676d17a-ec25-4a9a-9652-296114c72654 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:34.801619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4694cf9f-677b-4ef8-87e9-21c722eee0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:34.802236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694cf9f-677b-4ef8-87e9-21c722eee0c5 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:34.802515 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694cf9f-677b-4ef8-87e9-21c722eee0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:34.802788 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694cf9f-677b-4ef8-87e9-21c722eee0c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:34.803518 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4694cf9f-677b-4ef8-87e9-21c722eee0c5 None admin] RBAC: Adding query filter params (role.id=92533f29c1d54fe09591e6f2a906ac04, scope.project.id=181e5e0aca1b4b26890fff4fb7749311, user.id=98c373122e5c4b1a947b0dbd458f5b95) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:34.803832 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4694cf9f-677b-4ef8-87e9-21c722eee0c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:34.805548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4694cf9f-677b-4ef8-87e9-21c722eee0c5 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:34.811903 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 93/190] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:41:34 2022] GET /identity/v3/role_assignments?user.id=98c373122e5c4b1a947b0dbd458f5b95&scope.project.id=181e5e0aca1b4b26890fff4fb7749311&role.id=92533f29c1d54fe09591e6f2a906ac04 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:34.812549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4694cf9f-677b-4ef8-87e9-21c722eee0c5 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:34.828914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:34.838797 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fb0dc-b309-414b-bd45-77ebfcb1653d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:34.839532 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fb0dc-b309-414b-bd45-77ebfcb1653d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:34.839820 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fb0dc-b309-414b-bd45-77ebfcb1653d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:34.840116 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fb0dc-b309-414b-bd45-77ebfcb1653d None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:34.841714 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119fb0dc-b309-414b-bd45-77ebfcb1653d None admin] RBAC: Authorizing `identity:get_project(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:34.843268 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119fb0dc-b309-414b-bd45-77ebfcb1653d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:34.845081 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 98/191] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:34 2022] GET /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:34.850958 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fb0dc-b309-414b-bd45-77ebfcb1653d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:34.866271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:34.934746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4694cf9f-677b-4ef8-87e9-21c722eee0c5 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:34.974650 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b65082-41a8-4498-9532-f61286037729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:34.975275 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b65082-41a8-4498-9532-f61286037729 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:34.975556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b65082-41a8-4498-9532-f61286037729 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:34.975829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b65082-41a8-4498-9532-f61286037729 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:34.977016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fb0dc-b309-414b-bd45-77ebfcb1653d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:34.977148 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b65082-41a8-4498-9532-f61286037729 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:34.978530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b65082-41a8-4498-9532-f61286037729 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:35.005790 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 94/192] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 24 08:41:34 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:41:35.006730 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b65082-41a8-4498-9532-f61286037729 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:35.032853 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716fd74-3d07-4a4d-96db-e16587fbcd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:35.033832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716fd74-3d07-4a4d-96db-e16587fbcd55 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:35.034950 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716fd74-3d07-4a4d-96db-e16587fbcd55 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:35.035420 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716fd74-3d07-4a4d-96db-e16587fbcd55 None admin] PATH_INFO: `/v3/users/6a31812815c949f68ec0ce696ef2956b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:35.066554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716fd74-3d07-4a4d-96db-e16587fbcd55 None admin] RBAC: Authorizing `identity:get_user(user_id=6a31812815c949f68ec0ce696ef2956b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:35.069681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716fd74-3d07-4a4d-96db-e16587fbcd55 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:35.074450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 99/193] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:34 2022] GET /identity/v3/users/6a31812815c949f68ec0ce696ef2956b => generated 283 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:35.074450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716fd74-3d07-4a4d-96db-e16587fbcd55 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:35.123437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:35.163445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:35.173378 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b65082-41a8-4498-9532-f61286037729 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:35.206778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716fd74-3d07-4a4d-96db-e16587fbcd55 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:35.214529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e3239-690d-4499-9788-3d04bed791b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:35.215531 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e3239-690d-4499-9788-3d04bed791b9 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:35.215531 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e3239-690d-4499-9788-3d04bed791b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:35.215765 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e3239-690d-4499-9788-3d04bed791b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:35.216568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e3239-690d-4499-9788-3d04bed791b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:35.216881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e3239-690d-4499-9788-3d04bed791b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:35.218280 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e3239-690d-4499-9788-3d04bed791b9 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:35.227423 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 95/194] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:41:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:35.228321 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e3239-690d-4499-9788-3d04bed791b9 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:35.255656 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8685e-70ce-4519-b623-0f77f4ee8300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:35.256338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8685e-70ce-4519-b623-0f77f4ee8300 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:35.256769 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8685e-70ce-4519-b623-0f77f4ee8300 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:35.257044 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8685e-70ce-4519-b623-0f77f4ee8300 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:35.270790 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8685e-70ce-4519-b623-0f77f4ee8300 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:35.276277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-38b8685e-70ce-4519-b623-0f77f4ee8300 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:41:35.276277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:35.276277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:35.276277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:35.276277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:35.276277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:35.276277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:35.276277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:35.276277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:35.276277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:35.276277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:35.276277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:35.276277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:35.276277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:35.276277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:35.276277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:35.277406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:35.277406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:35.277406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:35.277406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:35.277406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:35.277406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:35.277406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:35.277406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:35.277406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:35.277406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:35.277406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:35.277406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:35.277406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:35.277406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:35.277406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:35.277406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:35.278372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:35.278372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:35.278372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:35.278372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:35.278372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:35.278372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:35.278372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:35.278372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:35.278372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:35.278372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:35.278372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:41:35.278372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:41:35.284030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 100/195] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:35.284030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8685e-70ce-4519-b623-0f77f4ee8300 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:35.371443 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e3239-690d-4499-9788-3d04bed791b9 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:35.415189 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:35.416439 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8685e-70ce-4519-b623-0f77f4ee8300 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:35.427031 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a880f6-3eb2-4b1d-9838-295e630984c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:35.428049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a880f6-3eb2-4b1d-9838-295e630984c9 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:35.428361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a880f6-3eb2-4b1d-9838-295e630984c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:35.428636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a880f6-3eb2-4b1d-9838-295e630984c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:35.429766 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a880f6-3eb2-4b1d-9838-295e630984c9 None admin] RBAC: Adding query filter params (role.id=92533f29c1d54fe09591e6f2a906ac04, scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d, user.id=98c373122e5c4b1a947b0dbd458f5b95) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:35.430401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a880f6-3eb2-4b1d-9838-295e630984c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:35.432051 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a880f6-3eb2-4b1d-9838-295e630984c9 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:35.440260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 96/196] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:41:35 2022] GET /identity/v3/role_assignments?user.id=98c373122e5c4b1a947b0dbd458f5b95&scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d&role.id=92533f29c1d54fe09591e6f2a906ac04 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:35.441457 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a880f6-3eb2-4b1d-9838-295e630984c9 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:35.457796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:35.475622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69349817-6787-4336-a5f6-374cc00355ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:35.476525 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69349817-6787-4336-a5f6-374cc00355ae None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:35.476884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69349817-6787-4336-a5f6-374cc00355ae None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:35.477107 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69349817-6787-4336-a5f6-374cc00355ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:35.635443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69349817-6787-4336-a5f6-374cc00355ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:35.637013 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69349817-6787-4336-a5f6-374cc00355ae None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:35.665588 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a880f6-3eb2-4b1d-9838-295e630984c9 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:35.703128 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b293d39-b4de-432c-a0d3-155878f067db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:35.703965 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b293d39-b4de-432c-a0d3-155878f067db None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:35.704782 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b293d39-b4de-432c-a0d3-155878f067db None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:35.705072 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b293d39-b4de-432c-a0d3-155878f067db None admin] PATH_INFO: `/v3/users/6a31812815c949f68ec0ce696ef2956b` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:35.706332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b293d39-b4de-432c-a0d3-155878f067db None admin] RBAC: Authorizing `identity:get_user(user_id=6a31812815c949f68ec0ce696ef2956b)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:35.708117 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b293d39-b4de-432c-a0d3-155878f067db None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:35.710349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 97/197] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:35 2022] GET /identity/v3/users/6a31812815c949f68ec0ce696ef2956b => generated 283 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:35.711531 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b293d39-b4de-432c-a0d3-155878f067db None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:35.796131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 101/198] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:41:35.796131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69349817-6787-4336-a5f6-374cc00355ae None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:35.801777 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:35.840518 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:35.875857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b293d39-b4de-432c-a0d3-155878f067db None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:35.921384 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2aec4-327a-4511-a0ff-a5da3f37c88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:35.921932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2aec4-327a-4511-a0ff-a5da3f37c88a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:35.922210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2aec4-327a-4511-a0ff-a5da3f37c88a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:35.922483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2aec4-327a-4511-a0ff-a5da3f37c88a None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:35.923519 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2aec4-327a-4511-a0ff-a5da3f37c88a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:35.923846 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2aec4-327a-4511-a0ff-a5da3f37c88a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:35.927996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2aec4-327a-4511-a0ff-a5da3f37c88a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:35.933229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69349817-6787-4336-a5f6-374cc00355ae None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:35.940385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 98/199] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:35.941030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2aec4-327a-4511-a0ff-a5da3f37c88a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:35.988054 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca023f6-e2e8-4ebb-b96b-39c06c4a93e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:35.989014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca023f6-e2e8-4ebb-b96b-39c06c4a93e8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:35.990044 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca023f6-e2e8-4ebb-b96b-39c06c4a93e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:35.990611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca023f6-e2e8-4ebb-b96b-39c06c4a93e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:36.072174 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2aec4-327a-4511-a0ff-a5da3f37c88a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:36.092515 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca023f6-e2e8-4ebb-b96b-39c06c4a93e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:36.094035 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca023f6-e2e8-4ebb-b96b-39c06c4a93e8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:36.104776 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:36.123284 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67cf94-ea5d-4692-b983-311220c54bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:36.124647 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67cf94-ea5d-4692-b983-311220c54bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:36.125519 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67cf94-ea5d-4692-b983-311220c54bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:36.126092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67cf94-ea5d-4692-b983-311220c54bf3 None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:36.127699 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67cf94-ea5d-4692-b983-311220c54bf3 None admin] RBAC: Authorizing `identity:get_project(project_id=9381518952174868a5bc99813df0342e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:36.130012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67cf94-ea5d-4692-b983-311220c54bf3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:36.132981 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 99/200] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:35 2022] GET /identity/v3/projects/9381518952174868a5bc99813df0342e => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:36.133999 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67cf94-ea5d-4692-b983-311220c54bf3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:36.140705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:36.236509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 102/201] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:41:36.236509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca023f6-e2e8-4ebb-b96b-39c06c4a93e8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:36.275415 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67cf94-ea5d-4692-b983-311220c54bf3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:36.325691 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318aff24-f3cd-49b2-bf76-026f48898614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:36.326362 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318aff24-f3cd-49b2-bf76-026f48898614 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:36.326718 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318aff24-f3cd-49b2-bf76-026f48898614 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:36.326996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318aff24-f3cd-49b2-bf76-026f48898614 None admin] PATH_INFO: `/v3/users/fc16f80c0e714a4282235bbecea23ffb` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:36.328781 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318aff24-f3cd-49b2-bf76-026f48898614 None admin] RBAC: Authorizing `identity:get_user(user_id=fc16f80c0e714a4282235bbecea23ffb)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:36.330645 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318aff24-f3cd-49b2-bf76-026f48898614 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:36.332749 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 100/202] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:36 2022] GET /identity/v3/users/fc16f80c0e714a4282235bbecea23ffb => generated 289 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:36.333562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318aff24-f3cd-49b2-bf76-026f48898614 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:36.373235 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca023f6-e2e8-4ebb-b96b-39c06c4a93e8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:36.410059 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:36.413502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:36.424114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df7fe0-359a-4198-8d85-516788d8c6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:36.425030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df7fe0-359a-4198-8d85-516788d8c6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:36.425312 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df7fe0-359a-4198-8d85-516788d8c6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:36.425585 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df7fe0-359a-4198-8d85-516788d8c6c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:36.426369 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df7fe0-359a-4198-8d85-516788d8c6c8 None admin] RBAC: Adding query filter params (role.id=b13ffa72344c4856901b1ba003e273fd, scope.project.id=9381518952174868a5bc99813df0342e, user.id=6a31812815c949f68ec0ce696ef2956b) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:36.426681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df7fe0-359a-4198-8d85-516788d8c6c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:36.428259 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df7fe0-359a-4198-8d85-516788d8c6c8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:36.439635 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 103/203] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:41:36 2022] GET /identity/v3/role_assignments?user.id=6a31812815c949f68ec0ce696ef2956b&scope.project.id=9381518952174868a5bc99813df0342e&role.id=b13ffa72344c4856901b1ba003e273fd => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:36.439839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df7fe0-359a-4198-8d85-516788d8c6c8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:36.512710 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318aff24-f3cd-49b2-bf76-026f48898614 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:36.586011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df7fe0-359a-4198-8d85-516788d8c6c8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:36.597190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977c3ae-ed99-4347-aad0-6093f99fb503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:36.600199 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977c3ae-ed99-4347-aad0-6093f99fb503 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:36.601045 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977c3ae-ed99-4347-aad0-6093f99fb503 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:36.603278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977c3ae-ed99-4347-aad0-6093f99fb503 None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:36.616863 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977c3ae-ed99-4347-aad0-6093f99fb503 None admin] RBAC: Authorizing `identity:get_project(project_id=9381518952174868a5bc99813df0342e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:36.620482 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977c3ae-ed99-4347-aad0-6093f99fb503 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:36.624469 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 101/204] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:36 2022] GET /identity/v3/projects/9381518952174868a5bc99813df0342e => generated 302 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:36.634829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977c3ae-ed99-4347-aad0-6093f99fb503 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:36.645128 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06bc47-1ba5-44e3-bd2f-0b861302c419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:36.645750 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06bc47-1ba5-44e3-bd2f-0b861302c419 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:36.646029 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06bc47-1ba5-44e3-bd2f-0b861302c419 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:36.646292 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06bc47-1ba5-44e3-bd2f-0b861302c419 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:36.724068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:36.737397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06bc47-1ba5-44e3-bd2f-0b861302c419 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:36.738961 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06bc47-1ba5-44e3-bd2f-0b861302c419 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:36.892945 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 104/205] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:41:36.899480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06bc47-1ba5-44e3-bd2f-0b861302c419 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:36.974316 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977c3ae-ed99-4347-aad0-6093f99fb503 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:37.016338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:37.031006 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06bc47-1ba5-44e3-bd2f-0b861302c419 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:37.054381 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:37.074316 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feac101-ee34-45dc-9456-8f46d4b8a04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:37.074970 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feac101-ee34-45dc-9456-8f46d4b8a04c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:37.075248 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feac101-ee34-45dc-9456-8f46d4b8a04c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:37.075518 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feac101-ee34-45dc-9456-8f46d4b8a04c None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:37.076727 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feac101-ee34-45dc-9456-8f46d4b8a04c None admin] RBAC: Authorizing `identity:get_project(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:37.078175 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feac101-ee34-45dc-9456-8f46d4b8a04c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:37.079837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 105/206] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:36 2022] GET /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:37.086232 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feac101-ee34-45dc-9456-8f46d4b8a04c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:37.088807 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b40ed6-0190-4008-adbc-6a5cd9cbd829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:37.089452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b40ed6-0190-4008-adbc-6a5cd9cbd829 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:37.089735 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b40ed6-0190-4008-adbc-6a5cd9cbd829 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:37.090012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b40ed6-0190-4008-adbc-6a5cd9cbd829 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:37.090766 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b40ed6-0190-4008-adbc-6a5cd9cbd829 None admin] RBAC: Adding query filter params (role.id=5915b1954b514151ab17ff5d531036e4, scope.project.id=9381518952174868a5bc99813df0342e, user.id=6a31812815c949f68ec0ce696ef2956b) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:37.091163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b40ed6-0190-4008-adbc-6a5cd9cbd829 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:37.092720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b40ed6-0190-4008-adbc-6a5cd9cbd829 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:37.109574 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 102/207] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:41:36 2022] GET /identity/v3/role_assignments?user.id=6a31812815c949f68ec0ce696ef2956b&scope.project.id=9381518952174868a5bc99813df0342e&role.id=5915b1954b514151ab17ff5d531036e4 => generated 259 bytes in 476 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:37.209724 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feac101-ee34-45dc-9456-8f46d4b8a04c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:37.249048 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482412d-8843-4b47-bfb1-d23d3bbcf14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:37.249729 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482412d-8843-4b47-bfb1-d23d3bbcf14c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:37.250015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482412d-8843-4b47-bfb1-d23d3bbcf14c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:37.250332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482412d-8843-4b47-bfb1-d23d3bbcf14c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:37.251130 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482412d-8843-4b47-bfb1-d23d3bbcf14c None admin] RBAC: Adding query filter params (role.id=7f4cf2414d134bd4ac30986436be0095, scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d, user.id=fc16f80c0e714a4282235bbecea23ffb) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:37.251439 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482412d-8843-4b47-bfb1-d23d3bbcf14c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:37.252909 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482412d-8843-4b47-bfb1-d23d3bbcf14c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:37.260257 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 106/208] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:41:37 2022] GET /identity/v3/role_assignments?user.id=fc16f80c0e714a4282235bbecea23ffb&scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d&role.id=7f4cf2414d134bd4ac30986436be0095 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:37.481984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87b69a-302f-4961-bc3e-6fd00d9bdb2b None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:37.482537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87b69a-302f-4961-bc3e-6fd00d9bdb2b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:37.482537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87b69a-302f-4961-bc3e-6fd00d9bdb2b None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:37.484277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 103/209] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:37.490104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7157bed-90cf-4b71-b433-919e981f5083 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:37.491008 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7157bed-90cf-4b71-b433-919e981f5083 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:37.491482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7157bed-90cf-4b71-b433-919e981f5083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:37.512698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e7157bed-90cf-4b71-b433-919e981f5083 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:37.514293 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e7157bed-90cf-4b71-b433-919e981f5083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:37.535284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:37.551869 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daa730-15a0-4889-9f0c-69fdc90d63f0 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:37.552286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daa730-15a0-4889-9f0c-69fdc90d63f0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:37.552619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daa730-15a0-4889-9f0c-69fdc90d63f0 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:37.555063 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 104/210] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:41:37.560704 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a29aa-0f09-482d-b22a-ec0407753000 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:37.561201 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a29aa-0f09-482d-b22a-ec0407753000 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:37.561632 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a29aa-0f09-482d-b22a-ec0407753000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:37.582101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-680a29aa-0f09-482d-b22a-ec0407753000 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:37.583357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-680a29aa-0f09-482d-b22a-ec0407753000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:37.618640 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 107/211] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:37.671689 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 105/212] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:37.910449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7157bed-90cf-4b71-b433-919e981f5083 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:37.962222 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a29aa-0f09-482d-b22a-ec0407753000 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:38.049332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7157bed-90cf-4b71-b433-919e981f5083 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:38.123409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a29aa-0f09-482d-b22a-ec0407753000 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:38.128498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded0ab3-2435-44ff-b0ea-8ee846ac018e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:38.129584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded0ab3-2435-44ff-b0ea-8ee846ac018e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:38.130078 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded0ab3-2435-44ff-b0ea-8ee846ac018e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:38.130555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded0ab3-2435-44ff-b0ea-8ee846ac018e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:38.132552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded0ab3-2435-44ff-b0ea-8ee846ac018e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:38.136036 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded0ab3-2435-44ff-b0ea-8ee846ac018e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:38.144199 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 108/213] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:41:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:38.150344 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded0ab3-2435-44ff-b0ea-8ee846ac018e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:38.164645 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:38.192520 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4feb061-784c-4110-84c5-48415400a580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:38.193580 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:38.194359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4feb061-784c-4110-84c5-48415400a580 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:38.195249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4feb061-784c-4110-84c5-48415400a580 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:38.196066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4feb061-784c-4110-84c5-48415400a580 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:38.216472 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4feb061-784c-4110-84c5-48415400a580 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:38.226568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-c4feb061-784c-4110-84c5-48415400a580 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:38.226568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:38.226568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:38.226568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:38.226568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:38.226568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:38.226568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:38.226568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:38.226568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:38.226568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:38.226568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:38.226568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:38.226568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:38.226568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:38.226568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:38.226568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:38.228334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:38.228334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:38.228334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:38.228334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:38.228334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:38.228334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:38.228334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:38.228334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:38.228334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:38.228334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:38.228334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:38.228334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:38.228334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:38.228334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:38.228334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:38.228334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:38.229897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:38.229897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:38.229897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:38.229897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:38.229897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:38.229897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:38.229897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:38.229897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:38.229897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:38.229897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:38.229897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:38.229897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:38.234311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 106/214] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:41:38.246146 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4feb061-784c-4110-84c5-48415400a580 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:38.310762 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded0ab3-2435-44ff-b0ea-8ee846ac018e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:38.361096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb1750-b99f-417e-9c54-37238dd001cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:38.361886 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb1750-b99f-417e-9c54-37238dd001cb None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:38.362188 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb1750-b99f-417e-9c54-37238dd001cb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:38.362560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb1750-b99f-417e-9c54-37238dd001cb None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:38.363360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb1750-b99f-417e-9c54-37238dd001cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:38.365404 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb1750-b99f-417e-9c54-37238dd001cb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:38.399887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 109/215] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 24 08:41:38 2022] POST /identity/v3/users => generated 297 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:41:38.440018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4feb061-784c-4110-84c5-48415400a580 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:38.485030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1ca8b-4ea1-42d6-9984-31596b0bd802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:38.485944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1ca8b-4ea1-42d6-9984-31596b0bd802 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:38.486061 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1ca8b-4ea1-42d6-9984-31596b0bd802 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:38.486385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1ca8b-4ea1-42d6-9984-31596b0bd802 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:38.486943 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1ca8b-4ea1-42d6-9984-31596b0bd802 None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:38.487250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1ca8b-4ea1-42d6-9984-31596b0bd802 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:38.488714 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1ca8b-4ea1-42d6-9984-31596b0bd802 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:38.504086 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 107/216] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:38.505825 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb1750-b99f-417e-9c54-37238dd001cb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:38.620635 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:38.648568 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb1750-b99f-417e-9c54-37238dd001cb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:38.692233 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307eb98-65de-4ee6-b541-f6fad3705e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:38.693020 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307eb98-65de-4ee6-b541-f6fad3705e3b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:38.693307 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307eb98-65de-4ee6-b541-f6fad3705e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:38.693632 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307eb98-65de-4ee6-b541-f6fad3705e3b None admin] PATH_INFO: `/v3/users/038f326d5c3e449ab63b21be5b787231` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:38.711954 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307eb98-65de-4ee6-b541-f6fad3705e3b None admin] RBAC: Authorizing `identity:get_user(user_id=038f326d5c3e449ab63b21be5b787231)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:38.714128 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307eb98-65de-4ee6-b541-f6fad3705e3b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:38.716730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 110/217] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:38 2022] GET /identity/v3/users/038f326d5c3e449ab63b21be5b787231 => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:38.722904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1ca8b-4ea1-42d6-9984-31596b0bd802 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:38.771748 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fbbc8-01eb-4f49-9630-33bd47792c4e None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:38.772061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fbbc8-01eb-4f49-9630-33bd47792c4e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:38.772346 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fbbc8-01eb-4f49-9630-33bd47792c4e None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:38.774568 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 111/218] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:41:38.781175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:38.781356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53f972-0922-4ff8-a844-b02633dc2019 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:38.781863 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53f972-0922-4ff8-a844-b02633dc2019 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:38.782366 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53f972-0922-4ff8-a844-b02633dc2019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:38.806332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-cf53f972-0922-4ff8-a844-b02633dc2019 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:38.807895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cf53f972-0922-4ff8-a844-b02633dc2019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:38.860371 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1ca8b-4ea1-42d6-9984-31596b0bd802 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:38.893738 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 112/219] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:38.901915 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d84a8-5d38-45fd-a1c9-773471c6a599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:38.902753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d84a8-5d38-45fd-a1c9-773471c6a599 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:38.903167 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d84a8-5d38-45fd-a1c9-773471c6a599 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:38.903580 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d84a8-5d38-45fd-a1c9-773471c6a599 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:38.986217 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6d84a8-5d38-45fd-a1c9-773471c6a599 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:38.987977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6d84a8-5d38-45fd-a1c9-773471c6a599 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:39.035407 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:39.112837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 108/220] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:41:39.119545 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53f972-0922-4ff8-a844-b02633dc2019 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:39.212593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:39.263709 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53f972-0922-4ff8-a844-b02633dc2019 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:39.318618 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbcb3a-22f5-4230-81cf-95d3f3e66116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:39.319291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbcb3a-22f5-4230-81cf-95d3f3e66116 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:39.319573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbcb3a-22f5-4230-81cf-95d3f3e66116 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:39.320495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbcb3a-22f5-4230-81cf-95d3f3e66116 None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:39.321921 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bbcb3a-22f5-4230-81cf-95d3f3e66116 None admin] RBAC: Authorizing `identity:get_project(project_id=9381518952174868a5bc99813df0342e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:39.323414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bbcb3a-22f5-4230-81cf-95d3f3e66116 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:39.325533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 113/221] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:39 2022] GET /identity/v3/projects/9381518952174868a5bc99813df0342e => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:39.331344 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d84a8-5d38-45fd-a1c9-773471c6a599 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:39.359181 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c322a7d-5789-4608-8a21-f4ecbfeb2dea None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:39.359531 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c322a7d-5789-4608-8a21-f4ecbfeb2dea None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:39.359722 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c322a7d-5789-4608-8a21-f4ecbfeb2dea None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:39.361547 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 114/222] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:41:39.366828 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442203ba-6fb4-4df9-b198-6c26330dfc25 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:39.367171 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442203ba-6fb4-4df9-b198-6c26330dfc25 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:39.367375 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442203ba-6fb4-4df9-b198-6c26330dfc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:39.396641 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-442203ba-6fb4-4df9-b198-6c26330dfc25 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:39.397755 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-442203ba-6fb4-4df9-b198-6c26330dfc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:39.477490 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d84a8-5d38-45fd-a1c9-773471c6a599 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:39.513540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 115/223] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:39.514864 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442203ba-6fb4-4df9-b198-6c26330dfc25 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:39.520071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ef833-50ee-4d48-b30d-0e0451647962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:39.521499 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ef833-50ee-4d48-b30d-0e0451647962 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:39.522008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ef833-50ee-4d48-b30d-0e0451647962 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:39.522541 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ef833-50ee-4d48-b30d-0e0451647962 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:39.524318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ef833-50ee-4d48-b30d-0e0451647962 None admin] RBAC: Adding query filter params (role.id=92533f29c1d54fe09591e6f2a906ac04, scope.project.id=9381518952174868a5bc99813df0342e, user.id=038f326d5c3e449ab63b21be5b787231) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:39.524900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ef833-50ee-4d48-b30d-0e0451647962 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:39.526727 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ef833-50ee-4d48-b30d-0e0451647962 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:39.535004 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 109/224] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:41:39 2022] GET /identity/v3/role_assignments?user.id=038f326d5c3e449ab63b21be5b787231&scope.project.id=9381518952174868a5bc99813df0342e&role.id=92533f29c1d54fe09591e6f2a906ac04 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:39.682274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:39.776817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442203ba-6fb4-4df9-b198-6c26330dfc25 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:39.830030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6883f6-f544-4b37-b493-7b64b9971c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:39.830669 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6883f6-f544-4b37-b493-7b64b9971c9b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:39.830947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6883f6-f544-4b37-b493-7b64b9971c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:39.831215 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6883f6-f544-4b37-b493-7b64b9971c9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:39.860005 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6883f6-f544-4b37-b493-7b64b9971c9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:39.865888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-3e6883f6-f544-4b37-b493-7b64b9971c9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:41:39.865888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:39.865888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:39.865888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:39.865888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:39.865888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:39.865888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:39.865888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:39.865888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:39.865888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:39.865888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:39.865888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:39.865888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:39.865888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:39.865888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:39.865888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:39.867353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:39.867353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:39.867353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:39.867353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:39.867353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:39.867353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:39.867353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:39.867353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:39.867353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:39.867353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:39.867353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:39.867353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:39.867353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:39.867353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:39.867353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:39.867353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:39.868284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:39.868284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:39.868284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:39.868284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:39.868284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:39.868284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:39.868284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:39.868284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:39.868284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:39.868284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:39.868284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:41:39.868284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:41:39.868284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 116/225] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:41:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:39.874389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ef833-50ee-4d48-b30d-0e0451647962 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:39.889259 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:39.975560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6883f6-f544-4b37-b493-7b64b9971c9b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:40.019460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ef833-50ee-4d48-b30d-0e0451647962 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:40.063355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba34f8-f34f-48ca-ac85-8ba93715809e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:40.064455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba34f8-f34f-48ca-ac85-8ba93715809e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:40.064959 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba34f8-f34f-48ca-ac85-8ba93715809e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:40.065438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba34f8-f34f-48ca-ac85-8ba93715809e None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:40.066345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba34f8-f34f-48ca-ac85-8ba93715809e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:40.066863 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba34f8-f34f-48ca-ac85-8ba93715809e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:40.068670 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba34f8-f34f-48ca-ac85-8ba93715809e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:40.079558 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 110/226] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:40.089509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba34f8-f34f-48ca-ac85-8ba93715809e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:40.206692 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6883f6-f544-4b37-b493-7b64b9971c9b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:40.248485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:40.256355 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:40.285585 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ceb88-63b5-4796-b85a-0fcb3bf85360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:40.286184 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ceb88-63b5-4796-b85a-0fcb3bf85360 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:40.286457 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ceb88-63b5-4796-b85a-0fcb3bf85360 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:40.286723 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ceb88-63b5-4796-b85a-0fcb3bf85360 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:40.312827 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ceb88-63b5-4796-b85a-0fcb3bf85360 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:40.317772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ef8ceb88-63b5-4796-b85a-0fcb3bf85360 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:41:40.317772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:40.317772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:40.317772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:40.317772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:40.317772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:40.317772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:40.317772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:40.317772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:40.317772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:40.317772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:40.317772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:40.317772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:40.317772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:40.317772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:40.317772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:40.318674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:40.318674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:40.318674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:40.318674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:40.318674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:40.318674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:40.318674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:40.318674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:40.318674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:40.318674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:40.318674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:40.318674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:40.318674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:40.318674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:40.318674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:40.318674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:40.320455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:40.320455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:40.320455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:40.320455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:40.320455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:40.320455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:40.320455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:40.320455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:40.320455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:40.320455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:40.320455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:41:40.320455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:41:40.320455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 117/227] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:41:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:41:40.327998 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ceb88-63b5-4796-b85a-0fcb3bf85360 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:40.338399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba34f8-f34f-48ca-ac85-8ba93715809e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:40.441656 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ceb88-63b5-4796-b85a-0fcb3bf85360 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:40.465451 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4dacc8-3ee0-4533-aca9-2ff009ae604d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:40.466120 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4dacc8-3ee0-4533-aca9-2ff009ae604d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:40.466406 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4dacc8-3ee0-4533-aca9-2ff009ae604d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:40.466682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4dacc8-3ee0-4533-aca9-2ff009ae604d None admin] PATH_INFO: `/v3/users/6a31812815c949f68ec0ce696ef2956b` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:40.472885 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4dacc8-3ee0-4533-aca9-2ff009ae604d None admin] RBAC: Authorizing `identity:get_user(user_id=6a31812815c949f68ec0ce696ef2956b)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:40.474380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4dacc8-3ee0-4533-aca9-2ff009ae604d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:40.476161 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 111/228] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:40 2022] GET /identity/v3/users/6a31812815c949f68ec0ce696ef2956b => generated 283 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:40.485107 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec496bcc-8731-4dbc-8fcb-9c02c4872b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:40.485731 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec496bcc-8731-4dbc-8fcb-9c02c4872b45 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:40.486002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec496bcc-8731-4dbc-8fcb-9c02c4872b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:40.486263 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec496bcc-8731-4dbc-8fcb-9c02c4872b45 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:40.486936 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec496bcc-8731-4dbc-8fcb-9c02c4872b45 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:40.487234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec496bcc-8731-4dbc-8fcb-9c02c4872b45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:40.488632 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec496bcc-8731-4dbc-8fcb-9c02c4872b45 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:40.489121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4dacc8-3ee0-4533-aca9-2ff009ae604d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:40.499260 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 118/229] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:41:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:40.511383 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec496bcc-8731-4dbc-8fcb-9c02c4872b45 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:40.577681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:40.645631 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec496bcc-8731-4dbc-8fcb-9c02c4872b45 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:40.683293 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcbeec-3dcf-4b66-ac9a-6a4111b6037c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:40.684041 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcbeec-3dcf-4b66-ac9a-6a4111b6037c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:40.684346 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcbeec-3dcf-4b66-ac9a-6a4111b6037c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:40.684613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcbeec-3dcf-4b66-ac9a-6a4111b6037c None admin] PATH_INFO: `/v3/users/6a31812815c949f68ec0ce696ef2956b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:40.685741 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fcbeec-3dcf-4b66-ac9a-6a4111b6037c None admin] RBAC: Authorizing `identity:get_user(user_id=6a31812815c949f68ec0ce696ef2956b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:40.687168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fcbeec-3dcf-4b66-ac9a-6a4111b6037c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:40.689069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 119/230] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:40 2022] GET /identity/v3/users/6a31812815c949f68ec0ce696ef2956b => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:40.694620 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcbeec-3dcf-4b66-ac9a-6a4111b6037c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:40.767450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4dacc8-3ee0-4533-aca9-2ff009ae604d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:40.842868 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5e608-4a96-4ea2-b4e7-b64f347be3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:40.843435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5e608-4a96-4ea2-b4e7-b64f347be3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:40.843710 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5e608-4a96-4ea2-b4e7-b64f347be3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:40.843976 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5e608-4a96-4ea2-b4e7-b64f347be3b9 None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:40.846193 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5e608-4a96-4ea2-b4e7-b64f347be3b9 None admin] RBAC: Authorizing `identity:get_project(project_id=9381518952174868a5bc99813df0342e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:40.847643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5e608-4a96-4ea2-b4e7-b64f347be3b9 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:40.849320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 112/231] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:40 2022] GET /identity/v3/projects/9381518952174868a5bc99813df0342e => generated 302 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:40.856558 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5e608-4a96-4ea2-b4e7-b64f347be3b9 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:40.860579 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:40.950910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcbeec-3dcf-4b66-ac9a-6a4111b6037c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:40.993748 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:41.017117 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb512af9-66d0-411d-a95b-a05a5ea8feaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:41.017764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb512af9-66d0-411d-a95b-a05a5ea8feaf None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:41.018043 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb512af9-66d0-411d-a95b-a05a5ea8feaf None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:41.018308 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb512af9-66d0-411d-a95b-a05a5ea8feaf None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:41.019442 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb512af9-66d0-411d-a95b-a05a5ea8feaf None admin] RBAC: Authorizing `identity:get_project(project_id=9381518952174868a5bc99813df0342e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:41.020933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb512af9-66d0-411d-a95b-a05a5ea8feaf None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:41.022852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 120/232] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:40 2022] GET /identity/v3/projects/9381518952174868a5bc99813df0342e => generated 302 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:41.024959 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af694154-1cf5-4e6f-95a7-c3a4ec78a60e None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:41.025243 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af694154-1cf5-4e6f-95a7-c3a4ec78a60e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:41.025563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af694154-1cf5-4e6f-95a7-c3a4ec78a60e None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:41.027051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 121/233] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:41.028279 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af694154-1cf5-4e6f-95a7-c3a4ec78a60e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:41.039501 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5e608-4a96-4ea2-b4e7-b64f347be3b9 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:41.076064 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926bca6a-ccf4-4c50-8e45-3d6beb45b1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:41.076767 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bca6a-ccf4-4c50-8e45-3d6beb45b1ab None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:41.077038 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bca6a-ccf4-4c50-8e45-3d6beb45b1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:41.077387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bca6a-ccf4-4c50-8e45-3d6beb45b1ab None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e/users/6a31812815c949f68ec0ce696ef2956b/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:41.079337 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926bca6a-ccf4-4c50-8e45-3d6beb45b1ab None admin] RBAC: Authorizing `identity:create_grant(project_id=9381518952174868a5bc99813df0342e, user_id=6a31812815c949f68ec0ce696ef2956b, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:41.080814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926bca6a-ccf4-4c50-8e45-3d6beb45b1ab None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:41.095746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 113/234] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:41:40 2022] PUT /identity/v3/projects/9381518952174868a5bc99813df0342e/users/6a31812815c949f68ec0ce696ef2956b/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:41:41.097289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d930b3-2309-4cd3-a7c4-63688c81acb5 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:41.097679 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d930b3-2309-4cd3-a7c4-63688c81acb5 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:41.098024 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d930b3-2309-4cd3-a7c4-63688c81acb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:41.119225 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f0d930b3-2309-4cd3-a7c4-63688c81acb5 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:41.120595 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f0d930b3-2309-4cd3-a7c4-63688c81acb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:41.164500 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af694154-1cf5-4e6f-95a7-c3a4ec78a60e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:41.194196 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:41.214595 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 114/235] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:41.215710 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34d602-aa73-4447-9ce9-82b8fa33fc82 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:41.215999 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34d602-aa73-4447-9ce9-82b8fa33fc82 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:41.216300 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34d602-aa73-4447-9ce9-82b8fa33fc82 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:41.222179 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 115/236] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:41.222491 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f2f8c-1238-4f08-8f14-ac24304e582c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:41.223168 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025046cf-348e-424f-b5a1-b3eec5b677cf None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:41.223454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025046cf-348e-424f-b5a1-b3eec5b677cf None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:41.223713 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025046cf-348e-424f-b5a1-b3eec5b677cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:41.224116 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f2f8c-1238-4f08-8f14-ac24304e582c None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:41.224585 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f2f8c-1238-4f08-8f14-ac24304e582c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:41.225092 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f2f8c-1238-4f08-8f14-ac24304e582c None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e/users/6a31812815c949f68ec0ce696ef2956b/roles/5915b1954b514151ab17ff5d531036e4` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:41.228889 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f2f8c-1238-4f08-8f14-ac24304e582c None admin] RBAC: Authorizing `identity:create_grant(project_id=9381518952174868a5bc99813df0342e, user_id=6a31812815c949f68ec0ce696ef2956b, role_id=5915b1954b514151ab17ff5d531036e4)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:41.231117 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f2f8c-1238-4f08-8f14-ac24304e582c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:41.250588 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-025046cf-348e-424f-b5a1-b3eec5b677cf None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:41.252124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-025046cf-348e-424f-b5a1-b3eec5b677cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:41.267582 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 122/237] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:41:41 2022] PUT /identity/v3/projects/9381518952174868a5bc99813df0342e/users/6a31812815c949f68ec0ce696ef2956b/roles/5915b1954b514151ab17ff5d531036e4 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:41:41.347952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 116/238] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:41.592744 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f2f8c-1238-4f08-8f14-ac24304e582c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:41.643424 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025046cf-348e-424f-b5a1-b3eec5b677cf None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:41.666222 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:41.749178 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f2f8c-1238-4f08-8f14-ac24304e582c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:41.770198 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025046cf-348e-424f-b5a1-b3eec5b677cf None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:41.797492 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cfbaa-217a-4724-a83c-b9e192fe4909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:41.798156 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cfbaa-217a-4724-a83c-b9e192fe4909 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:41.798441 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cfbaa-217a-4724-a83c-b9e192fe4909 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:41.800153 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cfbaa-217a-4724-a83c-b9e192fe4909 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:41.801656 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cfbaa-217a-4724-a83c-b9e192fe4909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:41.803121 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cfbaa-217a-4724-a83c-b9e192fe4909 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:41.806797 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 123/239] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:41:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:41.810177 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cfbaa-217a-4724-a83c-b9e192fe4909 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:41.826861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:41.841137 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a629e0e-534a-441b-9f99-64879383207d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:41.841787 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a629e0e-534a-441b-9f99-64879383207d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:41.842064 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a629e0e-534a-441b-9f99-64879383207d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:41.842332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a629e0e-534a-441b-9f99-64879383207d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:41.864394 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a629e0e-534a-441b-9f99-64879383207d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:41.869475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-6a629e0e-534a-441b-9f99-64879383207d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:41:41.869475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:41.869475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:41.869475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:41.869475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:41.869475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:41.869475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:41.869475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:41.869475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:41.869475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:41.869475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:41.869475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:41.869475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:41.869475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:41.869475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:41.869475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:41.870377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:41.870377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:41.870377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:41.870377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:41.870377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:41.870377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:41.870377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:41.870377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:41.870377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:41.870377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:41.870377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:41.870377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:41.870377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:41.870377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:41.870377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:41.870377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:41.871154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:41.871154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:41.871154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:41.871154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:41.871154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:41.871154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:41.871154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:41.871154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:41.871154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:41.871154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:41.871154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:41:41.871154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:41.871154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 117/240] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:41.877953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a629e0e-534a-441b-9f99-64879383207d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:41.979600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cfbaa-217a-4724-a83c-b9e192fe4909 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:41.996437 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:42.030278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224c249-fd1e-4b0e-a074-d67db6baaf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:42.030915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224c249-fd1e-4b0e-a074-d67db6baaf59 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:42.031194 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224c249-fd1e-4b0e-a074-d67db6baaf59 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:42.031467 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224c249-fd1e-4b0e-a074-d67db6baaf59 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:42.032172 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224c249-fd1e-4b0e-a074-d67db6baaf59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:42.033698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224c249-fd1e-4b0e-a074-d67db6baaf59 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:42.041844 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a629e0e-534a-441b-9f99-64879383207d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:42.068124 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 124/241] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 24 08:41:41 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:41:42.068124 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f896f4-0ec2-4ba1-bd19-89df0e8042d8 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:42.068124 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f896f4-0ec2-4ba1-bd19-89df0e8042d8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:42.068124 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f896f4-0ec2-4ba1-bd19-89df0e8042d8 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:42.068124 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 125/242] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:42.071541 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f8c2a-0772-4329-8a22-220b8aa015d1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:42.071830 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f8c2a-0772-4329-8a22-220b8aa015d1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:42.072120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f8c2a-0772-4329-8a22-220b8aa015d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:42.094139 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-fa8f8c2a-0772-4329-8a22-220b8aa015d1 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:42.095445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fa8f8c2a-0772-4329-8a22-220b8aa015d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:42.139304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84560c7-1c7a-410e-bfb9-fbb2a62f1d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:42.140389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84560c7-1c7a-410e-bfb9-fbb2a62f1d3a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:42.140948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84560c7-1c7a-410e-bfb9-fbb2a62f1d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:42.141432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84560c7-1c7a-410e-bfb9-fbb2a62f1d3a None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:42.142325 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84560c7-1c7a-410e-bfb9-fbb2a62f1d3a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:42.142851 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84560c7-1c7a-410e-bfb9-fbb2a62f1d3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:42.144472 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84560c7-1c7a-410e-bfb9-fbb2a62f1d3a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:42.162979 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 118/243] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:42.169058 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84560c7-1c7a-410e-bfb9-fbb2a62f1d3a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:42.184146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 126/244] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:42.264215 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:42.328709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84560c7-1c7a-410e-bfb9-fbb2a62f1d3a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:42.366826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932e1ab-3b3b-4062-b665-41ac337945f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:42.367673 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932e1ab-3b3b-4062-b665-41ac337945f7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:42.367950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932e1ab-3b3b-4062-b665-41ac337945f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:42.368253 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932e1ab-3b3b-4062-b665-41ac337945f7 None admin] PATH_INFO: `/v3/users/7a4e442f903a4d8ea52ce40a5821b9a5` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:42.386551 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932e1ab-3b3b-4062-b665-41ac337945f7 None admin] RBAC: Authorizing `identity:get_user(user_id=7a4e442f903a4d8ea52ce40a5821b9a5)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:42.388125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932e1ab-3b3b-4062-b665-41ac337945f7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:42.389891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 119/245] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:42 2022] GET /identity/v3/users/7a4e442f903a4d8ea52ce40a5821b9a5 => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:42.396405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f8c2a-0772-4329-8a22-220b8aa015d1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:42.498395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:42.500497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932e1ab-3b3b-4062-b665-41ac337945f7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:42.509286 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f8c2a-0772-4329-8a22-220b8aa015d1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:42.554310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b53157-1843-4d86-a255-5afdc1cd1649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:42.555294 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b53157-1843-4d86-a255-5afdc1cd1649 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:42.555790 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b53157-1843-4d86-a255-5afdc1cd1649 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:42.556897 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b53157-1843-4d86-a255-5afdc1cd1649 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:42.655731 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b53157-1843-4d86-a255-5afdc1cd1649 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:42.657254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b53157-1843-4d86-a255-5afdc1cd1649 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:42.665120 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932e1ab-3b3b-4062-b665-41ac337945f7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:42.668126 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:42.725006 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6f817-9283-4c5e-ade1-a1d50c12bd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:42.726110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6f817-9283-4c5e-ade1-a1d50c12bd1e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:42.726593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6f817-9283-4c5e-ade1-a1d50c12bd1e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:42.727060 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6f817-9283-4c5e-ade1-a1d50c12bd1e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:42.748081 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6f817-9283-4c5e-ade1-a1d50c12bd1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:42.753121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-8df6f817-9283-4c5e-ade1-a1d50c12bd1e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:42.753121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:42.753121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:42.753121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:42.753121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:42.753121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:42.753121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:42.753121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:42.753121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:42.753121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:42.753121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:42.753121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:42.753121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:42.753121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:42.753121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:42.753121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:42.754034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:42.754034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:42.754034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:42.754034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:42.754034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:42.754034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:42.754034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:42.754034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:42.754034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:42.754034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:42.754034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:42.754034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:42.754034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:42.754034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:42.754034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:42.754034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:42.754831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:42.754831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:42.754831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:42.754831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:42.754831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:42.754831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:42.754831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:42.754831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:42.754831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:42.754831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:42.754831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:42.754831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:42.758244 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 120/246] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:42.764161 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6f817-9283-4c5e-ade1-a1d50c12bd1e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:42.776570 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:42.790833 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 127/247] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:41:42.792979 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b53157-1843-4d86-a255-5afdc1cd1649 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:42.875031 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6f817-9283-4c5e-ade1-a1d50c12bd1e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:42.907978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b53157-1843-4d86-a255-5afdc1cd1649 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:42.915051 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22831501-bf2e-41ce-9eb7-4632a0142f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:42.915976 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22831501-bf2e-41ce-9eb7-4632a0142f9f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:42.916499 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22831501-bf2e-41ce-9eb7-4632a0142f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:42.916962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22831501-bf2e-41ce-9eb7-4632a0142f9f None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:42.917825 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22831501-bf2e-41ce-9eb7-4632a0142f9f None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:42.918414 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22831501-bf2e-41ce-9eb7-4632a0142f9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:42.920007 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22831501-bf2e-41ce-9eb7-4632a0142f9f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:42.930501 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 121/248] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:42.936339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22831501-bf2e-41ce-9eb7-4632a0142f9f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:42.947790 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a745ab4-310e-42c8-8712-b5fcaae9d265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:42.948715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a745ab4-310e-42c8-8712-b5fcaae9d265 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:42.948897 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a745ab4-310e-42c8-8712-b5fcaae9d265 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:42.949136 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a745ab4-310e-42c8-8712-b5fcaae9d265 None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:42.950346 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a745ab4-310e-42c8-8712-b5fcaae9d265 None admin] RBAC: Authorizing `identity:get_project(project_id=9381518952174868a5bc99813df0342e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:42.951975 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a745ab4-310e-42c8-8712-b5fcaae9d265 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:42.953927 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 128/249] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:42 2022] GET /identity/v3/projects/9381518952174868a5bc99813df0342e => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:42.960222 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a745ab4-310e-42c8-8712-b5fcaae9d265 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:42.968349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:43.036356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:43.047095 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22831501-bf2e-41ce-9eb7-4632a0142f9f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:43.082463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a745ab4-310e-42c8-8712-b5fcaae9d265 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:43.083944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91ebff-2950-47bb-917a-d216521abd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:43.084994 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91ebff-2950-47bb-917a-d216521abd47 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:43.085500 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91ebff-2950-47bb-917a-d216521abd47 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:43.085987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91ebff-2950-47bb-917a-d216521abd47 None admin] PATH_INFO: `/v3/users/038f326d5c3e449ab63b21be5b787231` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:43.087462 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91ebff-2950-47bb-917a-d216521abd47 None admin] RBAC: Authorizing `identity:get_user(user_id=038f326d5c3e449ab63b21be5b787231)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:43.089910 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91ebff-2950-47bb-917a-d216521abd47 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:43.092066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 122/250] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:42 2022] GET /identity/v3/users/038f326d5c3e449ab63b21be5b787231 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:43.100267 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91ebff-2950-47bb-917a-d216521abd47 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:43.124137 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc6b58-ce24-4d84-9a81-177c4a6d43f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:43.124829 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc6b58-ce24-4d84-9a81-177c4a6d43f7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:43.125163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc6b58-ce24-4d84-9a81-177c4a6d43f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:43.125429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc6b58-ce24-4d84-9a81-177c4a6d43f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:43.126166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc6b58-ce24-4d84-9a81-177c4a6d43f7 None admin] RBAC: Adding query filter params (role.id=7f4cf2414d134bd4ac30986436be0095, scope.project.id=9381518952174868a5bc99813df0342e, user.id=7a4e442f903a4d8ea52ce40a5821b9a5) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:43.126473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc6b58-ce24-4d84-9a81-177c4a6d43f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:43.128079 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc6b58-ce24-4d84-9a81-177c4a6d43f7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:43.135124 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 129/251] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:41:42 2022] GET /identity/v3/role_assignments?user.id=7a4e442f903a4d8ea52ce40a5821b9a5&scope.project.id=9381518952174868a5bc99813df0342e&role.id=7f4cf2414d134bd4ac30986436be0095 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:43.213982 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:43.221896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91ebff-2950-47bb-917a-d216521abd47 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:43.262232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725807b-197c-467d-89a3-2302fe5f044a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:43.263257 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725807b-197c-467d-89a3-2302fe5f044a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:43.263780 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725807b-197c-467d-89a3-2302fe5f044a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:43.264284 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725807b-197c-467d-89a3-2302fe5f044a None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:43.265780 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725807b-197c-467d-89a3-2302fe5f044a None admin] RBAC: Authorizing `identity:get_project(project_id=9381518952174868a5bc99813df0342e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:43.267534 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725807b-197c-467d-89a3-2302fe5f044a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:43.269908 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 123/252] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:43 2022] GET /identity/v3/projects/9381518952174868a5bc99813df0342e => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:43.276118 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc6b58-ce24-4d84-9a81-177c4a6d43f7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:43.410519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc6b58-ce24-4d84-9a81-177c4a6d43f7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:43.456996 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:43.467452 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc3c0a-7026-4a99-a83e-71a8a7cc9258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:43.468137 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc3c0a-7026-4a99-a83e-71a8a7cc9258 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:43.468424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc3c0a-7026-4a99-a83e-71a8a7cc9258 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:43.468756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc3c0a-7026-4a99-a83e-71a8a7cc9258 None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e/users/038f326d5c3e449ab63b21be5b787231/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:43.470958 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc3c0a-7026-4a99-a83e-71a8a7cc9258 None admin] RBAC: Authorizing `identity:create_grant(project_id=9381518952174868a5bc99813df0342e, user_id=038f326d5c3e449ab63b21be5b787231, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:43.472472 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc3c0a-7026-4a99-a83e-71a8a7cc9258 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:43.487752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 130/253] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:41:43 2022] PUT /identity/v3/projects/9381518952174868a5bc99813df0342e/users/038f326d5c3e449ab63b21be5b787231/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:41:43.774502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb92587-ffb9-4f6a-9d9c-c71e803657ff None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:43.775021 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb92587-ffb9-4f6a-9d9c-c71e803657ff None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:43.775438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb92587-ffb9-4f6a-9d9c-c71e803657ff None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:43.777285 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 124/254] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:43.782563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fa82b-9be4-4c18-a5b2-1eace1279e69 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:43.783044 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fa82b-9be4-4c18-a5b2-1eace1279e69 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:43.783148 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fa82b-9be4-4c18-a5b2-1eace1279e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:43.812240 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3e5fa82b-9be4-4c18-a5b2-1eace1279e69 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:43.813449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3e5fa82b-9be4-4c18-a5b2-1eace1279e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:43.884437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c5673-3bee-42f4-a8d9-f18b544cc941 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:43.885565 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c5673-3bee-42f4-a8d9-f18b544cc941 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:43.885565 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c5673-3bee-42f4-a8d9-f18b544cc941 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:43.886575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 125/255] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:43.892155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275b917-a47a-4d41-89fc-91c3ddf8ff44 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:43.892441 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275b917-a47a-4d41-89fc-91c3ddf8ff44 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:43.892713 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275b917-a47a-4d41-89fc-91c3ddf8ff44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:43.914652 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 131/256] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:43.937162 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1275b917-a47a-4d41-89fc-91c3ddf8ff44 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:43.938375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1275b917-a47a-4d41-89fc-91c3ddf8ff44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:44.063512 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 126/257] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:44.292695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fa82b-9be4-4c18-a5b2-1eace1279e69 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:44.399717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fa82b-9be4-4c18-a5b2-1eace1279e69 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:44.417960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:44.443071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8cd569-f3cd-4e5d-a0f7-257d7c41a3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:44.444057 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8cd569-f3cd-4e5d-a0f7-257d7c41a3ca None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:44.444589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8cd569-f3cd-4e5d-a0f7-257d7c41a3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:44.445107 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8cd569-f3cd-4e5d-a0f7-257d7c41a3ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:44.461720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8cd569-f3cd-4e5d-a0f7-257d7c41a3ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:44.466725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ed8cd569-f3cd-4e5d-a0f7-257d7c41a3ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:41:44.466725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:44.466725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:44.466725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:44.466725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:44.466725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:44.466725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:44.466725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:44.466725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:44.466725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:44.466725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:44.466725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:44.466725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:44.466725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:44.466725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:44.466725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:44.467596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:44.467596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:44.467596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:44.467596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:44.467596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:44.467596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:44.467596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:44.467596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:44.467596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:44.467596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:44.467596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:44.467596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:44.467596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:44.467596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:44.467596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:44.467596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:44.468415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:44.468415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:44.468415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:44.468415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:44.468415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:44.468415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:44.468415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:44.468415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:44.468415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:44.468415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:44.468415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:41:44.468415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:41:44.470644 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 132/258] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:41:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:44.477171 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275b917-a47a-4d41-89fc-91c3ddf8ff44 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:44.525464 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8cd569-f3cd-4e5d-a0f7-257d7c41a3ca None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:44.654529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:44.671910 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275b917-a47a-4d41-89fc-91c3ddf8ff44 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:44.766725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fae77b-6541-4d02-a483-4f3aeb2f4792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:44.767857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fae77b-6541-4d02-a483-4f3aeb2f4792 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:44.768168 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fae77b-6541-4d02-a483-4f3aeb2f4792 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:44.768446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fae77b-6541-4d02-a483-4f3aeb2f4792 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:44.769288 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fae77b-6541-4d02-a483-4f3aeb2f4792 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:44.769618 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fae77b-6541-4d02-a483-4f3aeb2f4792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:44.772125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fae77b-6541-4d02-a483-4f3aeb2f4792 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:44.787563 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 127/259] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:41:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:44.793735 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fae77b-6541-4d02-a483-4f3aeb2f4792 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:44.818839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8cd569-f3cd-4e5d-a0f7-257d7c41a3ca None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:44.928919 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f2b47-f512-4e9f-9247-b39ff7fe1874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:44.930622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f2b47-f512-4e9f-9247-b39ff7fe1874 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:44.931104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f2b47-f512-4e9f-9247-b39ff7fe1874 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:44.931567 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f2b47-f512-4e9f-9247-b39ff7fe1874 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:44.957274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f2b47-f512-4e9f-9247-b39ff7fe1874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:44.963063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-629f2b47-f512-4e9f-9247-b39ff7fe1874 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:41:44.963063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:44.963063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:44.963063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:44.963063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:44.963063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:44.963063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:44.963063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:44.963063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:44.963063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:44.963063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:44.963063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:44.963063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:44.963063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:44.963063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:44.963063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:44.963946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:44.963946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:44.963946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:44.963946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:44.963946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:44.963946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:44.963946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:44.963946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:44.963946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:44.963946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:44.963946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:44.963946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:44.963946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:44.963946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:44.963946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:44.963946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:44.964781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:44.964781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:44.964781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:44.964781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:44.964781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:44.964781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:44.964781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:44.964781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:44.964781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:44.964781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:44.964781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:41:44.964781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:41:44.974223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 133/260] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:41:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 449 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:44.979658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f2b47-f512-4e9f-9247-b39ff7fe1874 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:45.006207 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fae77b-6541-4d02-a483-4f3aeb2f4792 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:45.076832 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b2d9d-d324-4669-8b17-2258adbd5a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:45.080119 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:45.080256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b2d9d-d324-4669-8b17-2258adbd5a45 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:45.080256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b2d9d-d324-4669-8b17-2258adbd5a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:45.080256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b2d9d-d324-4669-8b17-2258adbd5a45 None admin] PATH_INFO: `/v3/users/6a31812815c949f68ec0ce696ef2956b` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:45.080627 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b2d9d-d324-4669-8b17-2258adbd5a45 None admin] RBAC: Authorizing `identity:get_user(user_id=6a31812815c949f68ec0ce696ef2956b)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:45.082224 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b2d9d-d324-4669-8b17-2258adbd5a45 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:45.089591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 128/261] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:44 2022] GET /identity/v3/users/6a31812815c949f68ec0ce696ef2956b => generated 283 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:45.094826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b2d9d-d324-4669-8b17-2258adbd5a45 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:45.134350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:45.206799 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f2b47-f512-4e9f-9247-b39ff7fe1874 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:45.279697 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b2d9d-d324-4669-8b17-2258adbd5a45 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:45.281375 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90621003-8bf2-4cff-9d99-b252de686d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:45.281996 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90621003-8bf2-4cff-9d99-b252de686d42 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:45.282276 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90621003-8bf2-4cff-9d99-b252de686d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:45.282545 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90621003-8bf2-4cff-9d99-b252de686d42 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:45.283774 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90621003-8bf2-4cff-9d99-b252de686d42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:45.283774 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90621003-8bf2-4cff-9d99-b252de686d42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:45.288141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90621003-8bf2-4cff-9d99-b252de686d42 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:45.303113 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 134/262] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:45.309116 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90621003-8bf2-4cff-9d99-b252de686d42 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:45.340155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32e51d-8cb0-44d0-9d4b-90c2b787bbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:45.341183 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32e51d-8cb0-44d0-9d4b-90c2b787bbf8 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:45.341652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32e51d-8cb0-44d0-9d4b-90c2b787bbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:45.342116 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32e51d-8cb0-44d0-9d4b-90c2b787bbf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:45.433210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32e51d-8cb0-44d0-9d4b-90c2b787bbf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:45.435021 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32e51d-8cb0-44d0-9d4b-90c2b787bbf8 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:45.462286 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:45.463875 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:45.465240 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90621003-8bf2-4cff-9d99-b252de686d42 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:45.533935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df538e87-b4cd-4ad2-aad1-bcabff4934ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:45.534593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df538e87-b4cd-4ad2-aad1-bcabff4934ec None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:45.534873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df538e87-b4cd-4ad2-aad1-bcabff4934ec None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:45.535141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df538e87-b4cd-4ad2-aad1-bcabff4934ec None admin] PATH_INFO: `/v3/users/6a31812815c949f68ec0ce696ef2956b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:45.536327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df538e87-b4cd-4ad2-aad1-bcabff4934ec None admin] RBAC: Authorizing `identity:get_user(user_id=6a31812815c949f68ec0ce696ef2956b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:45.537783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df538e87-b4cd-4ad2-aad1-bcabff4934ec None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:45.540297 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 135/263] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:45 2022] GET /identity/v3/users/6a31812815c949f68ec0ce696ef2956b => generated 283 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:45.545775 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df538e87-b4cd-4ad2-aad1-bcabff4934ec None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:45.556993 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 129/264] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:41:45.558798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c6a60-2f67-41a8-b5fb-d31954e35ee4 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:45.559229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c6a60-2f67-41a8-b5fb-d31954e35ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:45.559667 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c6a60-2f67-41a8-b5fb-d31954e35ee4 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:45.561147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 130/265] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:45.569306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c6a60-2f67-41a8-b5fb-d31954e35ee4 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:45.688636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c6a60-2f67-41a8-b5fb-d31954e35ee4 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:45.715336 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:45.733727 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50492b-f7a4-471f-9744-3fa10dd4dd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:45.734754 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50492b-f7a4-471f-9744-3fa10dd4dd2d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:45.735430 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50492b-f7a4-471f-9744-3fa10dd4dd2d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:45.736563 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50492b-f7a4-471f-9744-3fa10dd4dd2d None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:45.738327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50492b-f7a4-471f-9744-3fa10dd4dd2d None admin] RBAC: Authorizing `identity:get_project(project_id=9381518952174868a5bc99813df0342e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:45.746083 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50492b-f7a4-471f-9744-3fa10dd4dd2d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:45.752134 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 131/266] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:45 2022] GET /identity/v3/projects/9381518952174868a5bc99813df0342e => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:45.752134 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564bfeb-0eb4-4bc8-9bba-f2adf2e0d3dc None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:45.752134 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564bfeb-0eb4-4bc8-9bba-f2adf2e0d3dc None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:45.752134 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564bfeb-0eb4-4bc8-9bba-f2adf2e0d3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:45.778408 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e564bfeb-0eb4-4bc8-9bba-f2adf2e0d3dc None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:45.779617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e564bfeb-0eb4-4bc8-9bba-f2adf2e0d3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:45.795047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df538e87-b4cd-4ad2-aad1-bcabff4934ec None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:45.879717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d87c37-3392-4009-90fb-12964b954021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:45.881360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d87c37-3392-4009-90fb-12964b954021 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:45.881644 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d87c37-3392-4009-90fb-12964b954021 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:45.881915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d87c37-3392-4009-90fb-12964b954021 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:45.887830 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 132/267] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:45.888972 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564bfeb-0eb4-4bc8-9bba-f2adf2e0d3dc None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:45.983533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:46.007894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564bfeb-0eb4-4bc8-9bba-f2adf2e0d3dc None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:46.031414 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:46.051858 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c7666-e2f7-4590-bc53-6c2d6f54a945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:46.052599 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c7666-e2f7-4590-bc53-6c2d6f54a945 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:46.052882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c7666-e2f7-4590-bc53-6c2d6f54a945 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:46.053154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c7666-e2f7-4590-bc53-6c2d6f54a945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:46.054236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c7666-e2f7-4590-bc53-6c2d6f54a945 None admin] RBAC: Adding query filter params (role.id=5915b1954b514151ab17ff5d531036e4, scope.project.id=9381518952174868a5bc99813df0342e, user.id=6a31812815c949f68ec0ce696ef2956b) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:46.054553 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c7666-e2f7-4590-bc53-6c2d6f54a945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:46.056210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c7666-e2f7-4590-bc53-6c2d6f54a945 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:46.063575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 133/268] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:41:45 2022] GET /identity/v3/role_assignments?user.id=6a31812815c949f68ec0ce696ef2956b&scope.project.id=9381518952174868a5bc99813df0342e&role.id=5915b1954b514151ab17ff5d531036e4 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:46.065264 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c26e2-865b-4371-aef7-9e39caf8dd37 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:46.065546 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c26e2-865b-4371-aef7-9e39caf8dd37 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:46.065821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c26e2-865b-4371-aef7-9e39caf8dd37 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:46.067823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 134/269] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:46.074619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddef905-494c-4459-a8a2-ebc0fd5d3e75 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:46.074936 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddef905-494c-4459-a8a2-ebc0fd5d3e75 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:46.075164 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddef905-494c-4459-a8a2-ebc0fd5d3e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:46.090508 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d87c37-3392-4009-90fb-12964b954021 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:46.092057 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d87c37-3392-4009-90fb-12964b954021 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:46.096955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-dddef905-494c-4459-a8a2-ebc0fd5d3e75 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:46.098242 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dddef905-494c-4459-a8a2-ebc0fd5d3e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:46.185390 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 135/270] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:46.279509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005c537-9dbd-442b-8b75-6c634fe9885d None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:46.279801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005c537-9dbd-442b-8b75-6c634fe9885d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:46.279954 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005c537-9dbd-442b-8b75-6c634fe9885d None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:46.281391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 136/271] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:41:46.286824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c90ac-8b2f-46f1-8c0d-8708904897dd None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:46.287108 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c90ac-8b2f-46f1-8c0d-8708904897dd None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:46.287382 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c90ac-8b2f-46f1-8c0d-8708904897dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:46.307117 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 136/272] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 762 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:41:46.309566 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e08c90ac-8b2f-46f1-8c0d-8708904897dd None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:46.310761 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e08c90ac-8b2f-46f1-8c0d-8708904897dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:46.312690 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d87c37-3392-4009-90fb-12964b954021 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:46.424201 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 137/273] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:46.427953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c90ac-8b2f-46f1-8c0d-8708904897dd None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:46.441426 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:46.499117 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d87c37-3392-4009-90fb-12964b954021 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:46.508477 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:46.543588 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab88b0-a0ca-48bb-b2bd-a7410bc2b493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:46.544307 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab88b0-a0ca-48bb-b2bd-a7410bc2b493 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:46.544596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab88b0-a0ca-48bb-b2bd-a7410bc2b493 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:46.544871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab88b0-a0ca-48bb-b2bd-a7410bc2b493 None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:46.546056 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab88b0-a0ca-48bb-b2bd-a7410bc2b493 None admin] RBAC: Authorizing `identity:get_project(project_id=9381518952174868a5bc99813df0342e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:46.547529 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab88b0-a0ca-48bb-b2bd-a7410bc2b493 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:46.549447 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 137/274] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:46 2022] GET /identity/v3/projects/9381518952174868a5bc99813df0342e => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:46.556281 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a98c29-8105-4319-b5e2-1a2640a88e34 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:46.556561 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a98c29-8105-4319-b5e2-1a2640a88e34 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:46.556839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a98c29-8105-4319-b5e2-1a2640a88e34 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:46.558447 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 138/275] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:46.559710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a98c29-8105-4319-b5e2-1a2640a88e34 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:46.564661 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c90ac-8b2f-46f1-8c0d-8708904897dd None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:46.609510 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd73ec5-1fc3-4ee8-af27-eff961a6b8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:46.610148 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd73ec5-1fc3-4ee8-af27-eff961a6b8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:46.610432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd73ec5-1fc3-4ee8-af27-eff961a6b8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:46.610885 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd73ec5-1fc3-4ee8-af27-eff961a6b8a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:46.622121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd73ec5-1fc3-4ee8-af27-eff961a6b8a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:46.627340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-6dd73ec5-1fc3-4ee8-af27-eff961a6b8a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:46.627340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:46.627340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:46.627340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:46.627340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:46.627340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:46.627340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:46.627340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:46.627340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:46.627340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:46.627340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:46.627340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:46.627340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:46.627340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:46.627340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:46.627340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:46.628282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:46.628282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:46.628282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:46.628282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:46.628282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:46.628282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:46.628282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:46.628282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:46.628282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:46.628282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:46.628282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:46.628282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:46.628282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:46.628282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:46.628282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:46.628282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:46.629092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:46.629092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:46.629092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:46.629092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:46.629092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:46.629092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:46.629092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:46.629092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:46.629092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:46.629092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:46.629092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:46.629092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:46.629092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 138/276] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:46.635041 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf48e5-c0d7-4617-be00-842117923844 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:46.635041 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf48e5-c0d7-4617-be00-842117923844 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:46.635041 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf48e5-c0d7-4617-be00-842117923844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:46.650032 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-41cf48e5-c0d7-4617-be00-842117923844 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:46.651232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-41cf48e5-c0d7-4617-be00-842117923844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:46.693753 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a98c29-8105-4319-b5e2-1a2640a88e34 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:46.736443 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 139/277] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:46.736443 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf48e5-c0d7-4617-be00-842117923844 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:46.736694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf20510-2bf5-464d-96ad-f9bfafe30001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:46.736694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf20510-2bf5-464d-96ad-f9bfafe30001 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:46.736694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf20510-2bf5-464d-96ad-f9bfafe30001 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:46.736694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf20510-2bf5-464d-96ad-f9bfafe30001 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:46.736978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf20510-2bf5-464d-96ad-f9bfafe30001 None admin] RBAC: Adding query filter params (role.id=b13ffa72344c4856901b1ba003e273fd, scope.project.id=9381518952174868a5bc99813df0342e, user.id=6a31812815c949f68ec0ce696ef2956b) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:46.736978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf20510-2bf5-464d-96ad-f9bfafe30001 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:46.740205 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf20510-2bf5-464d-96ad-f9bfafe30001 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:46.748410 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:46.752310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 139/278] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:41:46 2022] GET /identity/v3/role_assignments?user.id=6a31812815c949f68ec0ce696ef2956b&scope.project.id=9381518952174868a5bc99813df0342e&role.id=b13ffa72344c4856901b1ba003e273fd => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:46.753353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf20510-2bf5-464d-96ad-f9bfafe30001 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:46.781512 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:46.931233 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf48e5-c0d7-4617-be00-842117923844 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:46.936627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf20510-2bf5-464d-96ad-f9bfafe30001 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:46.988753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1c2c4-715b-4efa-939d-5159e13b0654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:46.989404 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1c2c4-715b-4efa-939d-5159e13b0654 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:46.989681 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1c2c4-715b-4efa-939d-5159e13b0654 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:46.989950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1c2c4-715b-4efa-939d-5159e13b0654 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:46.991069 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1c2c4-715b-4efa-939d-5159e13b0654 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:46.992618 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2e82f-c896-4f26-82e5-c1b9b9d901a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:46.992755 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1c2c4-715b-4efa-939d-5159e13b0654 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:46.993380 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2e82f-c896-4f26-82e5-c1b9b9d901a0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:46.993632 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2e82f-c896-4f26-82e5-c1b9b9d901a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:46.993932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2e82f-c896-4f26-82e5-c1b9b9d901a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:46.994558 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 140/279] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:41:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:46.994694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2e82f-c896-4f26-82e5-c1b9b9d901a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:46.994940 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2e82f-c896-4f26-82e5-c1b9b9d901a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:46.995176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1c2c4-715b-4efa-939d-5159e13b0654 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:46.996447 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2e82f-c896-4f26-82e5-c1b9b9d901a0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:47.005690 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 140/280] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:47.006550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2e82f-c896-4f26-82e5-c1b9b9d901a0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:47.084708 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:47.123683 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1c2c4-715b-4efa-939d-5159e13b0654 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:47.125162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2e82f-c896-4f26-82e5-c1b9b9d901a0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:47.125301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:47.160352 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cbcf4-739b-40d8-8f6e-6d79f18d8396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:47.160961 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cbcf4-739b-40d8-8f6e-6d79f18d8396 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:47.161244 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cbcf4-739b-40d8-8f6e-6d79f18d8396 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:47.161514 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cbcf4-739b-40d8-8f6e-6d79f18d8396 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:47.161809 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e443d0d-dda9-4581-80ea-ec85d0c41151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:47.162164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5cbcf4-739b-40d8-8f6e-6d79f18d8396 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:47.162442 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e443d0d-dda9-4581-80ea-ec85d0c41151 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:47.162727 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e443d0d-dda9-4581-80ea-ec85d0c41151 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:47.162993 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e443d0d-dda9-4581-80ea-ec85d0c41151 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:47.163681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5cbcf4-739b-40d8-8f6e-6d79f18d8396 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:47.173988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e443d0d-dda9-4581-80ea-ec85d0c41151 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:47.181486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-6e443d0d-dda9-4581-80ea-ec85d0c41151 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:41:47.181486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:47.181486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:47.181486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:47.181486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:47.181486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:47.181486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:47.181486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:47.181486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:47.181486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:47.181486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:47.181486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:47.181486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:47.181486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:47.181486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:47.181486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:47.182525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:47.182525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:47.182525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:47.182525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:47.182525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:47.182525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:47.182525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:47.182525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:47.182525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:47.182525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:47.182525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:47.182525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:47.182525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:47.182525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:47.182525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:47.182525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:47.188353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:47.188353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:47.188353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:47.188353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:47.188353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:47.188353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:47.188353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:47.188353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:47.188353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:47.188353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:47.188353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:41:47.188353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:47.188353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 141/281] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:47.188353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e443d0d-dda9-4581-80ea-ec85d0c41151 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:47.203474 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 141/282] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 24 08:41:47 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:41:47.203474 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cbcf4-739b-40d8-8f6e-6d79f18d8396 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:47.330261 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e443d0d-dda9-4581-80ea-ec85d0c41151 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:47.363891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cbcf4-739b-40d8-8f6e-6d79f18d8396 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:47.390632 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:47.392477 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c1b7b-6dbf-4a8b-b815-ed29f13f4ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:47.394647 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c1b7b-6dbf-4a8b-b815-ed29f13f4ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:47.394647 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c1b7b-6dbf-4a8b-b815-ed29f13f4ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:47.394855 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c1b7b-6dbf-4a8b-b815-ed29f13f4ef8 None admin] PATH_INFO: `/v3/users/448f33ef29994926b886d8eb62844a65` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:47.418517 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5654c0-57de-4dbd-8931-af854c3b892e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:47.418989 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5654c0-57de-4dbd-8931-af854c3b892e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:47.419445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5654c0-57de-4dbd-8931-af854c3b892e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:47.419854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5654c0-57de-4dbd-8931-af854c3b892e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:47.424392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c1b7b-6dbf-4a8b-b815-ed29f13f4ef8 None admin] RBAC: Authorizing `identity:get_user(user_id=448f33ef29994926b886d8eb62844a65)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:47.427483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c1b7b-6dbf-4a8b-b815-ed29f13f4ef8 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:47.431471 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 142/283] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:47 2022] GET /identity/v3/users/448f33ef29994926b886d8eb62844a65 => generated 293 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:47.432702 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c1b7b-6dbf-4a8b-b815-ed29f13f4ef8 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:47.438361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5654c0-57de-4dbd-8931-af854c3b892e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:47.447145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ec5654c0-57de-4dbd-8931-af854c3b892e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:47.447145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:47.447145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:47.447145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:47.447145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:47.447145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:47.447145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:47.447145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:47.447145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:47.447145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:47.447145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:47.447145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:47.447145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:47.447145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:47.447145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:47.447145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:47.448131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:47.448131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:47.448131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:47.448131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:47.448131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:47.448131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:47.448131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:47.448131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:47.448131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:47.448131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:47.448131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:47.448131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:47.448131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:47.448131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:47.448131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:47.448131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:47.448941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:47.448941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:47.448941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:47.448941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:47.448941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:47.448941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:47.448941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:47.448941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:47.448941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:47.448941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:47.448941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:47.448941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:41:47.448941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 142/284] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:47.450708 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5654c0-57de-4dbd-8931-af854c3b892e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:47.467762 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:47.575263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c1b7b-6dbf-4a8b-b815-ed29f13f4ef8 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:47.594790 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5654c0-57de-4dbd-8931-af854c3b892e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:47.623750 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d071dc6-7444-4c70-bc94-f4b184e81ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:47.624705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d071dc6-7444-4c70-bc94-f4b184e81ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:47.625432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d071dc6-7444-4c70-bc94-f4b184e81ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:47.625937 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d071dc6-7444-4c70-bc94-f4b184e81ac8 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:47.626851 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d071dc6-7444-4c70-bc94-f4b184e81ac8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:47.627397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d071dc6-7444-4c70-bc94-f4b184e81ac8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:47.629543 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d071dc6-7444-4c70-bc94-f4b184e81ac8 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:47.639678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ad3ab-c999-453d-a5b2-f6930a5199e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:47.640273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 143/285] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:47.640643 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ad3ab-c999-453d-a5b2-f6930a5199e8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:47.640928 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ad3ab-c999-453d-a5b2-f6930a5199e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:47.641200 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ad3ab-c999-453d-a5b2-f6930a5199e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:47.641354 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d071dc6-7444-4c70-bc94-f4b184e81ac8 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:47.642051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ad3ab-c999-453d-a5b2-f6930a5199e8 None admin] RBAC: Adding query filter params (role.id=92533f29c1d54fe09591e6f2a906ac04, user.id=448f33ef29994926b886d8eb62844a65, scope.system=all) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:47.642718 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ad3ab-c999-453d-a5b2-f6930a5199e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:47.644784 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ad3ab-c999-453d-a5b2-f6930a5199e8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:47.652657 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 143/286] 10.222.0.67 () {56 vars in 1308 bytes} [Wed Aug 24 08:41:47 2022] GET /identity/v3/role_assignments?user.id=448f33ef29994926b886d8eb62844a65&scope.system=all&role.id=92533f29c1d54fe09591e6f2a906ac04 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:47.654702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ad3ab-c999-453d-a5b2-f6930a5199e8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:47.765335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:47.795540 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:47.813807 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d071dc6-7444-4c70-bc94-f4b184e81ac8 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:47.854579 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90409383-8d53-4663-bd18-d072ba34fa31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:47.855634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90409383-8d53-4663-bd18-d072ba34fa31 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:47.856264 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90409383-8d53-4663-bd18-d072ba34fa31 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:47.856773 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90409383-8d53-4663-bd18-d072ba34fa31 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:47.857655 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90409383-8d53-4663-bd18-d072ba34fa31 None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:47.859139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90409383-8d53-4663-bd18-d072ba34fa31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:47.860941 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90409383-8d53-4663-bd18-d072ba34fa31 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:47.872607 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 144/287] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:47.876141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ad3ab-c999-453d-a5b2-f6930a5199e8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:47.880046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90409383-8d53-4663-bd18-d072ba34fa31 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:47.917032 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b7c0a-20da-4d04-90f0-ae49f152ce26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:47.917729 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b7c0a-20da-4d04-90f0-ae49f152ce26 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:47.918001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b7c0a-20da-4d04-90f0-ae49f152ce26 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:47.918272 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b7c0a-20da-4d04-90f0-ae49f152ce26 None admin] PATH_INFO: `/v3/users/7a4e442f903a4d8ea52ce40a5821b9a5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:47.919491 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b7c0a-20da-4d04-90f0-ae49f152ce26 None admin] RBAC: Authorizing `identity:get_user(user_id=7a4e442f903a4d8ea52ce40a5821b9a5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:47.921039 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b7c0a-20da-4d04-90f0-ae49f152ce26 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:47.922676 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 144/288] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:47 2022] GET /identity/v3/users/7a4e442f903a4d8ea52ce40a5821b9a5 => generated 297 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:47.928638 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b7c0a-20da-4d04-90f0-ae49f152ce26 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:48.010921 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90409383-8d53-4663-bd18-d072ba34fa31 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:48.050242 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b2534-e542-4082-8bb7-d9fdac2c4de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:48.051385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b2534-e542-4082-8bb7-d9fdac2c4de6 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:48.052547 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b2534-e542-4082-8bb7-d9fdac2c4de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:48.053183 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b2534-e542-4082-8bb7-d9fdac2c4de6 None admin] PATH_INFO: `/v3/users/038f326d5c3e449ab63b21be5b787231` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:48.057957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b2534-e542-4082-8bb7-d9fdac2c4de6 None admin] RBAC: Authorizing `identity:get_user(user_id=038f326d5c3e449ab63b21be5b787231)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:48.059781 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b2534-e542-4082-8bb7-d9fdac2c4de6 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:48.061990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 145/289] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:47 2022] GET /identity/v3/users/038f326d5c3e449ab63b21be5b787231 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:48.067725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b2534-e542-4082-8bb7-d9fdac2c4de6 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:48.100461 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:48.135621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b7c0a-20da-4d04-90f0-ae49f152ce26 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:48.160573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:48.214097 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b2534-e542-4082-8bb7-d9fdac2c4de6 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:48.250781 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0e27d-cf3b-4dfc-ad2f-3f2ecd69e295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:48.251850 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0e27d-cf3b-4dfc-ad2f-3f2ecd69e295 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:48.252472 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0e27d-cf3b-4dfc-ad2f-3f2ecd69e295 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:48.252890 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0e27d-cf3b-4dfc-ad2f-3f2ecd69e295 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:48.254100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c64a89-254c-4678-ae98-668b97ff021b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:48.254765 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c64a89-254c-4678-ae98-668b97ff021b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:48.255049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c64a89-254c-4678-ae98-668b97ff021b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:48.255325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c64a89-254c-4678-ae98-668b97ff021b None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:48.260908 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c64a89-254c-4678-ae98-668b97ff021b None admin] RBAC: Authorizing `identity:get_project(project_id=9381518952174868a5bc99813df0342e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:48.262466 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c64a89-254c-4678-ae98-668b97ff021b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:48.264169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 145/290] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:47 2022] GET /identity/v3/projects/9381518952174868a5bc99813df0342e => generated 302 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:48.270087 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c64a89-254c-4678-ae98-668b97ff021b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:48.340795 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0e27d-cf3b-4dfc-ad2f-3f2ecd69e295 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:48.342778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0e27d-cf3b-4dfc-ad2f-3f2ecd69e295 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:48.375358 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:48.487170 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c64a89-254c-4678-ae98-668b97ff021b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:48.534621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e4410-2262-4cc9-b9c5-957898d65be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:48.535352 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e4410-2262-4cc9-b9c5-957898d65be2 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:48.535629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e4410-2262-4cc9-b9c5-957898d65be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:48.535897 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e4410-2262-4cc9-b9c5-957898d65be2 None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e/users/7a4e442f903a4d8ea52ce40a5821b9a5/roles/7f4cf2414d134bd4ac30986436be0095` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:48.537999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e4410-2262-4cc9-b9c5-957898d65be2 None admin] RBAC: Authorizing `identity:create_grant(project_id=9381518952174868a5bc99813df0342e, user_id=7a4e442f903a4d8ea52ce40a5821b9a5, role_id=7f4cf2414d134bd4ac30986436be0095)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:48.541441 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e4410-2262-4cc9-b9c5-957898d65be2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:48.555539 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 146/291] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:41:48.563656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0e27d-cf3b-4dfc-ad2f-3f2ecd69e295 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:48.573392 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 146/292] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:41:48 2022] PUT /identity/v3/projects/9381518952174868a5bc99813df0342e/users/7a4e442f903a4d8ea52ce40a5821b9a5/roles/7f4cf2414d134bd4ac30986436be0095 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:41:48.725265 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0e27d-cf3b-4dfc-ad2f-3f2ecd69e295 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:48.756476 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:48.788179 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4e7ab-2afa-403b-8976-dcdcbc874955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:48.789340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4e7ab-2afa-403b-8976-dcdcbc874955 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:48.789664 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4e7ab-2afa-403b-8976-dcdcbc874955 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:48.790125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4e7ab-2afa-403b-8976-dcdcbc874955 None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:48.792216 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e4e7ab-2afa-403b-8976-dcdcbc874955 None admin] RBAC: Authorizing `identity:get_project(project_id=9381518952174868a5bc99813df0342e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:48.794782 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e4e7ab-2afa-403b-8976-dcdcbc874955 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:48.798236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 147/293] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:48 2022] GET /identity/v3/projects/9381518952174868a5bc99813df0342e => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:48.806556 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e4410-2262-4cc9-b9c5-957898d65be2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:48.856971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:48.926744 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e4410-2262-4cc9-b9c5-957898d65be2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:48.965130 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23d11f-d2d1-4670-995c-e9588cf457de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:48.965748 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23d11f-d2d1-4670-995c-e9588cf457de None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:48.966030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23d11f-d2d1-4670-995c-e9588cf457de None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:48.966306 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23d11f-d2d1-4670-995c-e9588cf457de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:48.967039 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca23d11f-d2d1-4670-995c-e9588cf457de None admin] RBAC: Adding query filter params (role.id=92533f29c1d54fe09591e6f2a906ac04, scope.project.id=9381518952174868a5bc99813df0342e, user.id=038f326d5c3e449ab63b21be5b787231) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:48.967348 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca23d11f-d2d1-4670-995c-e9588cf457de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:48.968807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca23d11f-d2d1-4670-995c-e9588cf457de None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:48.975263 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 147/294] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:41:48 2022] GET /identity/v3/role_assignments?user.id=038f326d5c3e449ab63b21be5b787231&scope.project.id=9381518952174868a5bc99813df0342e&role.id=92533f29c1d54fe09591e6f2a906ac04 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:49.612582 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599fa31-138e-4c4b-bf55-e9893c63ef4d None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:49.613263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599fa31-138e-4c4b-bf55-e9893c63ef4d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:49.613263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599fa31-138e-4c4b-bf55-e9893c63ef4d None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:49.614523 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 148/295] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:49.624363 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d91fc-32ba-4eca-8fee-b08eba595aa3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:49.625114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d91fc-32ba-4eca-8fee-b08eba595aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:49.625114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d91fc-32ba-4eca-8fee-b08eba595aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:49.647175 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b19d91fc-32ba-4eca-8fee-b08eba595aa3 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:49.648988 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b19d91fc-32ba-4eca-8fee-b08eba595aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:49.728184 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 148/296] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:49.753167 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b616c4-c9d3-433b-93de-6552c207088c None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:49.753799 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b616c4-c9d3-433b-93de-6552c207088c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:49.754294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b616c4-c9d3-433b-93de-6552c207088c None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:49.755843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 149/297] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:49.762517 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350df1e8-f8f1-443b-9fea-ffcb0fc6dfe7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:49.763095 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350df1e8-f8f1-443b-9fea-ffcb0fc6dfe7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:49.763567 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350df1e8-f8f1-443b-9fea-ffcb0fc6dfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:49.782762 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-350df1e8-f8f1-443b-9fea-ffcb0fc6dfe7 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:49.784307 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-350df1e8-f8f1-443b-9fea-ffcb0fc6dfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:49.817879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:49.868846 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 149/298] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:50.008467 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b616c4-c9d3-433b-93de-6552c207088c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:50.117038 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b616c4-c9d3-433b-93de-6552c207088c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:50.125873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700b96d-d398-4d97-9247-fb82d0d34651 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:50.126138 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700b96d-d398-4d97-9247-fb82d0d34651 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:50.126360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700b96d-d398-4d97-9247-fb82d0d34651 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:50.128087 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 150/299] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:50.133770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53898af4-f8c7-4e9e-a041-71285d4bbc33 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:50.134330 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53898af4-f8c7-4e9e-a041-71285d4bbc33 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:50.134613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53898af4-f8c7-4e9e-a041-71285d4bbc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:50.158445 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2879d-6e3b-44af-8786-7eb63bd2bdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:50.158677 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-53898af4-f8c7-4e9e-a041-71285d4bbc33 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:50.158900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2879d-6e3b-44af-8786-7eb63bd2bdec None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:50.159159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2879d-6e3b-44af-8786-7eb63bd2bdec None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:50.159375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2879d-6e3b-44af-8786-7eb63bd2bdec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:50.159527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-53898af4-f8c7-4e9e-a041-71285d4bbc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:50.170275 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2879d-6e3b-44af-8786-7eb63bd2bdec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:50.175330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-d5c2879d-6e3b-44af-8786-7eb63bd2bdec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:41:50.175330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:50.175330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:50.175330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:50.175330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:50.175330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:50.175330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:50.175330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:50.175330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:50.175330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:50.175330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:50.175330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:50.175330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:50.175330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:50.175330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:50.175330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:50.176345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:50.176345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:50.176345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:50.176345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:50.176345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:50.176345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:50.176345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:50.176345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:50.176345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:50.176345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:50.176345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:50.176345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:50.176345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:50.176345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:50.176345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:50.176345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:50.177239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:50.177239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:50.177239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:50.177239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:50.177239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:50.177239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:50.177239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:50.177239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:50.177239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:50.177239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:50.177239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:41:50.177239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:50.177239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 150/300] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:50.179563 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2879d-6e3b-44af-8786-7eb63bd2bdec None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:50.223161 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:50.235768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 151/301] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:50.241042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53898af4-f8c7-4e9e-a041-71285d4bbc33 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:50.310583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2879d-6e3b-44af-8786-7eb63bd2bdec None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:50.362362 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c18d14-2a10-4520-9be9-42c9c6d7d2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:50.363009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c18d14-2a10-4520-9be9-42c9c6d7d2c5 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:50.363293 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c18d14-2a10-4520-9be9-42c9c6d7d2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:50.363569 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c18d14-2a10-4520-9be9-42c9c6d7d2c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:50.365103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c18d14-2a10-4520-9be9-42c9c6d7d2c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:50.367911 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c18d14-2a10-4520-9be9-42c9c6d7d2c5 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:50.369730 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 151/302] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:41:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:50.375280 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c18d14-2a10-4520-9be9-42c9c6d7d2c5 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:50.380203 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:50.391346 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53898af4-f8c7-4e9e-a041-71285d4bbc33 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:50.449761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f6844-cc9d-4d52-bf72-b7370baad8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:50.450433 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f6844-cc9d-4d52-bf72-b7370baad8bc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:50.450765 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f6844-cc9d-4d52-bf72-b7370baad8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:50.451037 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f6844-cc9d-4d52-bf72-b7370baad8bc None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:50.452313 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f6844-cc9d-4d52-bf72-b7370baad8bc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:50.453015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f6844-cc9d-4d52-bf72-b7370baad8bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:50.455475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f6844-cc9d-4d52-bf72-b7370baad8bc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:50.468284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 152/303] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:50.479471 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f6844-cc9d-4d52-bf72-b7370baad8bc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:50.516672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c18d14-2a10-4520-9be9-42c9c6d7d2c5 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:50.527342 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:50.562331 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef54c8-89c2-492b-9d8c-87ddf21ce0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:50.562944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef54c8-89c2-492b-9d8c-87ddf21ce0f6 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:50.563232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef54c8-89c2-492b-9d8c-87ddf21ce0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:50.563499 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef54c8-89c2-492b-9d8c-87ddf21ce0f6 None admin] PATH_INFO: `/v3/groups` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:50.564168 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef54c8-89c2-492b-9d8c-87ddf21ce0f6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:50.565619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef54c8-89c2-492b-9d8c-87ddf21ce0f6 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:50.596301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 152/304] 10.222.0.67 () {60 vars in 1134 bytes} [Wed Aug 24 08:41:50 2022] POST /identity/v3/groups => generated 229 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:41:50.597650 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef54c8-89c2-492b-9d8c-87ddf21ce0f6 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:50.613987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f6844-cc9d-4d52-bf72-b7370baad8bc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:50.667611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e62ac3-3170-4637-84f9-5df7129bc6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:50.668350 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e62ac3-3170-4637-84f9-5df7129bc6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:50.668739 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e62ac3-3170-4637-84f9-5df7129bc6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:50.669817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e62ac3-3170-4637-84f9-5df7129bc6b7 None admin] PATH_INFO: `/v3/users/e2836a70b2044fa9b6f965349929496e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:50.690823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e62ac3-3170-4637-84f9-5df7129bc6b7 None admin] RBAC: Authorizing `identity:get_user(user_id=e2836a70b2044fa9b6f965349929496e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:50.692472 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e62ac3-3170-4637-84f9-5df7129bc6b7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:50.695366 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 153/305] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:50 2022] GET /identity/v3/users/e2836a70b2044fa9b6f965349929496e => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:50.701447 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e62ac3-3170-4637-84f9-5df7129bc6b7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:50.737754 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:50.759574 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef54c8-89c2-492b-9d8c-87ddf21ce0f6 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:50.812704 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e71a1-c6c2-4546-9d4a-df1c96034b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:50.813409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e71a1-c6c2-4546-9d4a-df1c96034b73 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:50.813699 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e71a1-c6c2-4546-9d4a-df1c96034b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:50.813982 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e71a1-c6c2-4546-9d4a-df1c96034b73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:50.833578 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e71a1-c6c2-4546-9d4a-df1c96034b73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:50.839283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-6e6e71a1-c6c2-4546-9d4a-df1c96034b73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:50.839283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:50.839283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:50.839283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:50.839283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:50.839283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:50.839283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:50.839283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:50.839283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:50.839283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:50.839283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:50.839283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:50.839283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:50.839283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:50.839283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:50.839283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:50.840633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:50.840633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:50.840633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:50.840633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:50.840633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:50.840633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:50.840633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:50.840633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:50.840633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:50.840633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:50.840633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:50.840633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:50.840633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:50.840633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:50.840633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:50.840633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:50.841725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:50.841725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:50.841725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:50.841725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:50.841725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:50.841725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:50.841725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:50.841725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:50.841725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:50.841725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:50.841725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:50.841725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:50.841725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 153/306] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:50.849078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e71a1-c6c2-4546-9d4a-df1c96034b73 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:50.889695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:50.892338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e62ac3-3170-4637-84f9-5df7129bc6b7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:50.942528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ef565-5245-4cbe-9221-7fe51f87591b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:50.943604 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ef565-5245-4cbe-9221-7fe51f87591b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:50.944533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ef565-5245-4cbe-9221-7fe51f87591b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:50.945437 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ef565-5245-4cbe-9221-7fe51f87591b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:50.946242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ef565-5245-4cbe-9221-7fe51f87591b None admin] RBAC: Adding query filter params (role.id=7f4cf2414d134bd4ac30986436be0095, user.id=e2836a70b2044fa9b6f965349929496e, scope.system=all) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:50.946556 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ef565-5245-4cbe-9221-7fe51f87591b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:50.948034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ef565-5245-4cbe-9221-7fe51f87591b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:50.953932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 154/307] 10.222.0.67 () {56 vars in 1308 bytes} [Wed Aug 24 08:41:50 2022] GET /identity/v3/role_assignments?user.id=e2836a70b2044fa9b6f965349929496e&scope.system=all&role.id=7f4cf2414d134bd4ac30986436be0095 => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:50.954998 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059b43a-5c57-43c8-b989-9e0dd915bcdb None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:50.955276 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059b43a-5c57-43c8-b989-9e0dd915bcdb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:50.955541 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059b43a-5c57-43c8-b989-9e0dd915bcdb None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:50.957000 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 155/308] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:50.962742 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e50916-f73b-4920-82b7-8d14fc3cbfe6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:50.963107 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e50916-f73b-4920-82b7-8d14fc3cbfe6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:50.963351 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e50916-f73b-4920-82b7-8d14fc3cbfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:50.983719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e7e50916-f73b-4920-82b7-8d14fc3cbfe6 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:50.985141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e7e50916-f73b-4920-82b7-8d14fc3cbfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:50.996836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e71a1-c6c2-4546-9d4a-df1c96034b73 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:51.052405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042747c-a9bd-45bc-aef8-8da7ce99655c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:51.053530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042747c-a9bd-45bc-aef8-8da7ce99655c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:51.055631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042747c-a9bd-45bc-aef8-8da7ce99655c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:51.056204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042747c-a9bd-45bc-aef8-8da7ce99655c None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:51.057485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042747c-a9bd-45bc-aef8-8da7ce99655c None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:51.058286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042747c-a9bd-45bc-aef8-8da7ce99655c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:51.060723 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042747c-a9bd-45bc-aef8-8da7ce99655c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:51.072133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 154/309] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:51.079217 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042747c-a9bd-45bc-aef8-8da7ce99655c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:51.088138 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 156/310] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:51.187415 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:51.198569 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042747c-a9bd-45bc-aef8-8da7ce99655c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:51.236031 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cecab1-110a-4682-9a00-ef936eca9bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:51.236767 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cecab1-110a-4682-9a00-ef936eca9bcb None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:51.237048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cecab1-110a-4682-9a00-ef936eca9bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:51.237393 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cecab1-110a-4682-9a00-ef936eca9bcb None admin] PATH_INFO: `/v3/users/448f33ef29994926b886d8eb62844a65` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:51.238610 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cecab1-110a-4682-9a00-ef936eca9bcb None admin] RBAC: Authorizing `identity:get_user(user_id=448f33ef29994926b886d8eb62844a65)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:51.240247 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cecab1-110a-4682-9a00-ef936eca9bcb None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:51.242231 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 155/311] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:51 2022] GET /identity/v3/users/448f33ef29994926b886d8eb62844a65 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:51.248121 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e50916-f73b-4920-82b7-8d14fc3cbfe6 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:51.258038 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:51.356376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e50916-f73b-4920-82b7-8d14fc3cbfe6 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:51.363554 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cecab1-110a-4682-9a00-ef936eca9bcb None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:51.395123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec54dc2-8de2-4112-9db9-85731e127175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:51.395926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec54dc2-8de2-4112-9db9-85731e127175 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:51.396554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec54dc2-8de2-4112-9db9-85731e127175 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:51.397053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec54dc2-8de2-4112-9db9-85731e127175 None admin] PATH_INFO: `/v3/system/users/448f33ef29994926b886d8eb62844a65/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:51.398839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec54dc2-8de2-4112-9db9-85731e127175 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=448f33ef29994926b886d8eb62844a65, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:51.400485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec54dc2-8de2-4112-9db9-85731e127175 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:51.415342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 157/312] 10.222.0.67 () {58 vars in 1334 bytes} [Wed Aug 24 08:41:51 2022] PUT /identity/v3/system/users/448f33ef29994926b886d8eb62844a65/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:41:51.503171 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cecab1-110a-4682-9a00-ef936eca9bcb None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:51.565204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690009b6-0739-471e-9428-9c8f1b5939b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:51.566670 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690009b6-0739-471e-9428-9c8f1b5939b3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:51.567349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690009b6-0739-471e-9428-9c8f1b5939b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:51.568055 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690009b6-0739-471e-9428-9c8f1b5939b3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:51.590070 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690009b6-0739-471e-9428-9c8f1b5939b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:51.601403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-690009b6-0739-471e-9428-9c8f1b5939b3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:41:51.601403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:51.601403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:51.601403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:51.601403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:51.601403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:51.601403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:51.601403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:51.601403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:51.601403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:51.601403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:51.601403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:51.601403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:51.601403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:51.601403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:51.601403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:51.603361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:51.603361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:51.603361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:51.603361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:51.603361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:51.603361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:51.603361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:51.603361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:51.603361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:51.603361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:51.603361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:51.603361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:51.603361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:51.603361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:51.603361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:51.603361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:51.605229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:51.605229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:51.605229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:51.605229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:51.605229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:51.605229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:51.605229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:51.605229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:51.605229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:51.605229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:51.605229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:41:51.605229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:51.608811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 156/313] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:41:51.618651 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec54dc2-8de2-4112-9db9-85731e127175 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:51.781198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:51.821207 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec54dc2-8de2-4112-9db9-85731e127175 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:51.880496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76194aea-c6cd-40e9-a427-bb5e384d647d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:51.881559 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76194aea-c6cd-40e9-a427-bb5e384d647d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:51.882102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76194aea-c6cd-40e9-a427-bb5e384d647d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:51.882619 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76194aea-c6cd-40e9-a427-bb5e384d647d None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:51.884002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76194aea-c6cd-40e9-a427-bb5e384d647d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:51.884573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76194aea-c6cd-40e9-a427-bb5e384d647d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:51.887199 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76194aea-c6cd-40e9-a427-bb5e384d647d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:51.903913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 158/314] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:51.913412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690009b6-0739-471e-9428-9c8f1b5939b3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:51.949913 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:52.092189 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690009b6-0739-471e-9428-9c8f1b5939b3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:52.128158 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb67e1e-3b6d-4e0a-ac70-693db4b0c59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:52.128807 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb67e1e-3b6d-4e0a-ac70-693db4b0c59f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:52.129088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb67e1e-3b6d-4e0a-ac70-693db4b0c59f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:52.129369 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb67e1e-3b6d-4e0a-ac70-693db4b0c59f None admin] PATH_INFO: `/v3/users/7a4e442f903a4d8ea52ce40a5821b9a5` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:52.130539 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb67e1e-3b6d-4e0a-ac70-693db4b0c59f None admin] RBAC: Authorizing `identity:get_user(user_id=7a4e442f903a4d8ea52ce40a5821b9a5)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:52.132540 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb67e1e-3b6d-4e0a-ac70-693db4b0c59f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:52.134395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 157/315] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:51 2022] GET /identity/v3/users/7a4e442f903a4d8ea52ce40a5821b9a5 => generated 297 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:52.140445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76194aea-c6cd-40e9-a427-bb5e384d647d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:52.250699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76194aea-c6cd-40e9-a427-bb5e384d647d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:52.281486 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:52.286745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2304ef-edbf-47df-9a6b-bbc7b5c76801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:52.287658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2304ef-edbf-47df-9a6b-bbc7b5c76801 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:52.288030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2304ef-edbf-47df-9a6b-bbc7b5c76801 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:52.288896 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2304ef-edbf-47df-9a6b-bbc7b5c76801 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:52.362520 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2304ef-edbf-47df-9a6b-bbc7b5c76801 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:52.363919 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2304ef-edbf-47df-9a6b-bbc7b5c76801 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:52.472815 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 159/316] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:41:52.478278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb67e1e-3b6d-4e0a-ac70-693db4b0c59f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:52.581380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:52.598687 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb67e1e-3b6d-4e0a-ac70-693db4b0c59f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:52.636495 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c70b9a-a5bb-44f5-91e4-63d745454930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:52.637165 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c70b9a-a5bb-44f5-91e4-63d745454930 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:52.637449 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c70b9a-a5bb-44f5-91e4-63d745454930 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:52.637724 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c70b9a-a5bb-44f5-91e4-63d745454930 None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:52.639176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c70b9a-a5bb-44f5-91e4-63d745454930 None admin] RBAC: Authorizing `identity:get_project(project_id=9381518952174868a5bc99813df0342e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:52.640746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c70b9a-a5bb-44f5-91e4-63d745454930 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:52.642898 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 158/317] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:52 2022] GET /identity/v3/projects/9381518952174868a5bc99813df0342e => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:52.649816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2304ef-edbf-47df-9a6b-bbc7b5c76801 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:52.688240 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:52.769087 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2304ef-edbf-47df-9a6b-bbc7b5c76801 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:52.809446 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6578a-09cb-4fcd-80f5-51cb4ac425c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:52.811150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6578a-09cb-4fcd-80f5-51cb4ac425c2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:52.811438 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6578a-09cb-4fcd-80f5-51cb4ac425c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:52.811713 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6578a-09cb-4fcd-80f5-51cb4ac425c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:52.812489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6578a-09cb-4fcd-80f5-51cb4ac425c2 None admin] RBAC: Adding query filter params (role.id=7f4cf2414d134bd4ac30986436be0095, scope.project.id=9381518952174868a5bc99813df0342e, user.id=7a4e442f903a4d8ea52ce40a5821b9a5) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:52.812801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6578a-09cb-4fcd-80f5-51cb4ac425c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:52.814252 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6578a-09cb-4fcd-80f5-51cb4ac425c2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:52.821772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 160/318] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:41:52 2022] GET /identity/v3/role_assignments?user.id=7a4e442f903a4d8ea52ce40a5821b9a5&scope.project.id=9381518952174868a5bc99813df0342e&role.id=7f4cf2414d134bd4ac30986436be0095 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:52.938620 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a9cd5-fd61-4ad9-871b-2b3769851bba None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:52.939609 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a9cd5-fd61-4ad9-871b-2b3769851bba None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:52.940082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a9cd5-fd61-4ad9-871b-2b3769851bba None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:52.942150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 159/319] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:52.949083 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342e87c-825f-4c05-a6db-7cecc9c86d10 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:52.949387 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342e87c-825f-4c05-a6db-7cecc9c86d10 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:52.949654 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342e87c-825f-4c05-a6db-7cecc9c86d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:52.969316 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2342e87c-825f-4c05-a6db-7cecc9c86d10 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:52.970517 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2342e87c-825f-4c05-a6db-7cecc9c86d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:53.048228 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f7e13-4760-4d6a-885a-760094e61388 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:53.048840 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f7e13-4760-4d6a-885a-760094e61388 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:53.049330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f7e13-4760-4d6a-885a-760094e61388 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:53.051147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 160/320] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:53.057138 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c337991-7959-4e57-a23c-1764677cd309 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:53.057440 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c337991-7959-4e57-a23c-1764677cd309 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:53.057607 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c337991-7959-4e57-a23c-1764677cd309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:53.059425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 161/321] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:53.082951 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-3c337991-7959-4e57-a23c-1764677cd309 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:53.084209 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3c337991-7959-4e57-a23c-1764677cd309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:53.180159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 161/322] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:53.336753 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342e87c-825f-4c05-a6db-7cecc9c86d10 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:53.369775 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:53.442339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c337991-7959-4e57-a23c-1764677cd309 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:53.453222 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342e87c-825f-4c05-a6db-7cecc9c86d10 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:53.496537 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b337f8-c00a-418c-a956-8c5afc2db0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:53.497177 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b337f8-c00a-418c-a956-8c5afc2db0ac None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:53.497455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b337f8-c00a-418c-a956-8c5afc2db0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:53.497720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b337f8-c00a-418c-a956-8c5afc2db0ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:53.498815 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b337f8-c00a-418c-a956-8c5afc2db0ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:53.500283 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b337f8-c00a-418c-a956-8c5afc2db0ac None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:53.502066 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 162/323] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:41:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:53.507296 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b337f8-c00a-418c-a956-8c5afc2db0ac None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:53.538899 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:53.567078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c337991-7959-4e57-a23c-1764677cd309 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:53.606249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204da51-3e9a-4c5c-861c-9c834fe4bcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:53.606786 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204da51-3e9a-4c5c-861c-9c834fe4bcf1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:53.607061 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204da51-3e9a-4c5c-861c-9c834fe4bcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:53.607331 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204da51-3e9a-4c5c-861c-9c834fe4bcf1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:53.610304 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:53.617962 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b337f8-c00a-418c-a956-8c5afc2db0ac None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:53.618297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204da51-3e9a-4c5c-861c-9c834fe4bcf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:53.623018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-6204da51-3e9a-4c5c-861c-9c834fe4bcf1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:41:53.623018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:53.623018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:53.623018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:53.623018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:53.623018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:53.623018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:53.623018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:53.623018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:53.623018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:53.623018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:53.623018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:53.623018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:53.623018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:53.623018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:53.623018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:53.623938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:53.623938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:53.623938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:53.623938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:53.623938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:53.623938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:53.623938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:53.623938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:53.623938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:53.623938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:53.623938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:53.623938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:53.623938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:53.623938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:53.623938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:53.623938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:53.624760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:53.624760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:53.624760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:53.624760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:53.624760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:53.624760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:53.624760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:53.624760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:53.624760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:53.624760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:53.624760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:41:53.624760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:53.624760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 162/324] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:41:53.626881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d65da-ff36-414a-a3f9-9750266cd99a None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:53.627166 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d65da-ff36-414a-a3f9-9750266cd99a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:53.627438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d65da-ff36-414a-a3f9-9750266cd99a None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:53.628849 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 163/325] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:53.633031 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d65da-ff36-414a-a3f9-9750266cd99a None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:53.653865 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea91c0-3e36-4463-87d2-4936396360b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:53.654481 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea91c0-3e36-4463-87d2-4936396360b1 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:53.654761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea91c0-3e36-4463-87d2-4936396360b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:53.655033 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea91c0-3e36-4463-87d2-4936396360b1 None admin] PATH_INFO: `/v3/groups` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:53.655662 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea91c0-3e36-4463-87d2-4936396360b1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:53.657149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea91c0-3e36-4463-87d2-4936396360b1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:53.674303 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 163/326] 10.222.0.67 () {60 vars in 1134 bytes} [Wed Aug 24 08:41:53 2022] POST /identity/v3/groups => generated 226 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:41:53.676073 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b09bfc-419b-4699-a980-8c98e8bf7bf2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:53.676378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b09bfc-419b-4699-a980-8c98e8bf7bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:53.676648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b09bfc-419b-4699-a980-8c98e8bf7bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:53.704205 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d9b09bfc-419b-4699-a980-8c98e8bf7bf2 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:53.705421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d9b09bfc-419b-4699-a980-8c98e8bf7bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:53.772301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d65da-ff36-414a-a3f9-9750266cd99a None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:53.788053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 164/327] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:53.818008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28816238-11d0-4af7-8f64-c817d4b5f429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:53.818008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28816238-11d0-4af7-8f64-c817d4b5f429 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:53.818312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28816238-11d0-4af7-8f64-c817d4b5f429 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:53.818689 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28816238-11d0-4af7-8f64-c817d4b5f429 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:53.819524 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28816238-11d0-4af7-8f64-c817d4b5f429 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:53.820020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28816238-11d0-4af7-8f64-c817d4b5f429 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:53.821896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28816238-11d0-4af7-8f64-c817d4b5f429 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:53.830913 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 164/328] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:53.836489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b09bfc-419b-4699-a980-8c98e8bf7bf2 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:53.938763 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:53.961138 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b09bfc-419b-4699-a980-8c98e8bf7bf2 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:54.002841 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e4034-0809-4951-8e61-6ec2d789fd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:54.003488 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e4034-0809-4951-8e61-6ec2d789fd84 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:54.003764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e4034-0809-4951-8e61-6ec2d789fd84 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:54.004031 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e4034-0809-4951-8e61-6ec2d789fd84 None admin] PATH_INFO: `/v3/users/e2836a70b2044fa9b6f965349929496e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:54.005245 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e4034-0809-4951-8e61-6ec2d789fd84 None admin] RBAC: Authorizing `identity:get_user(user_id=e2836a70b2044fa9b6f965349929496e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:54.006685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e4034-0809-4951-8e61-6ec2d789fd84 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:54.008675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 165/329] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:53 2022] GET /identity/v3/users/e2836a70b2044fa9b6f965349929496e => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:54.014569 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28816238-11d0-4af7-8f64-c817d4b5f429 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:54.025454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:54.074799 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e4034-0809-4951-8e61-6ec2d789fd84 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:54.158197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28816238-11d0-4af7-8f64-c817d4b5f429 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:54.200632 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61236e4b-7557-46a9-ace1-79a031d11762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:54.201240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61236e4b-7557-46a9-ace1-79a031d11762 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:54.201517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61236e4b-7557-46a9-ace1-79a031d11762 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:54.201789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61236e4b-7557-46a9-ace1-79a031d11762 None admin] PATH_INFO: `/v3/system/users/e2836a70b2044fa9b6f965349929496e/roles/7f4cf2414d134bd4ac30986436be0095` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:54.203259 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61236e4b-7557-46a9-ace1-79a031d11762 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e2836a70b2044fa9b6f965349929496e, role_id=7f4cf2414d134bd4ac30986436be0095)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:54.204677 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61236e4b-7557-46a9-ace1-79a031d11762 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:54.215545 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 165/330] 10.222.0.67 () {58 vars in 1334 bytes} [Wed Aug 24 08:41:54 2022] PUT /identity/v3/system/users/e2836a70b2044fa9b6f965349929496e/roles/7f4cf2414d134bd4ac30986436be0095 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:41:54.229343 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e4034-0809-4951-8e61-6ec2d789fd84 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:54.280362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b10b59-18e7-40f5-9ba5-d3e357f0aafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:54.281464 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b10b59-18e7-40f5-9ba5-d3e357f0aafa None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:54.282029 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b10b59-18e7-40f5-9ba5-d3e357f0aafa None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:54.282480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b10b59-18e7-40f5-9ba5-d3e357f0aafa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:54.301287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b10b59-18e7-40f5-9ba5-d3e357f0aafa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:54.310600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-59b10b59-18e7-40f5-9ba5-d3e357f0aafa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:54.310600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:54.310600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:54.310600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:54.310600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:54.310600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:54.310600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:54.310600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:54.310600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:54.310600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:54.310600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:54.310600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:54.310600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:54.310600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:54.310600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:54.310600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:54.311818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:54.311818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:54.311818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:54.311818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:54.311818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:54.311818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:54.311818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:54.311818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:54.311818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:54.311818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:54.311818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:54.311818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:54.311818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:54.311818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:54.311818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:54.311818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:54.313065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:54.313065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:54.313065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:54.313065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:54.313065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:54.313065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:54.313065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:54.313065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:54.313065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:54.313065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:54.313065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:54.313065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:41:54.315840 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 166/331] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:41:54.322130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61236e4b-7557-46a9-ace1-79a031d11762 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:54.435171 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:54.481457 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61236e4b-7557-46a9-ace1-79a031d11762 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:54.530043 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce7c61-1449-4d61-b643-f22dc7b544aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:54.530681 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce7c61-1449-4d61-b643-f22dc7b544aa None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:54.530960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce7c61-1449-4d61-b643-f22dc7b544aa None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:54.531237 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce7c61-1449-4d61-b643-f22dc7b544aa None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:54.531925 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce7c61-1449-4d61-b643-f22dc7b544aa None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:54.532298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce7c61-1449-4d61-b643-f22dc7b544aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:54.533696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce7c61-1449-4d61-b643-f22dc7b544aa None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:54.549321 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 166/332] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:54.555860 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b10b59-18e7-40f5-9ba5-d3e357f0aafa None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:54.588251 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:54.719332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b10b59-18e7-40f5-9ba5-d3e357f0aafa None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:54.774618 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec29efd-1799-4d8b-b276-010395fb7092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:54.775294 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec29efd-1799-4d8b-b276-010395fb7092 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:54.775573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec29efd-1799-4d8b-b276-010395fb7092 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:54.775849 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec29efd-1799-4d8b-b276-010395fb7092 None admin] PATH_INFO: `/v3/users/448f33ef29994926b886d8eb62844a65` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:54.777099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec29efd-1799-4d8b-b276-010395fb7092 None admin] RBAC: Authorizing `identity:get_user(user_id=448f33ef29994926b886d8eb62844a65)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:54.778562 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec29efd-1799-4d8b-b276-010395fb7092 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:54.780432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 167/333] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:54 2022] GET /identity/v3/users/448f33ef29994926b886d8eb62844a65 => generated 293 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:54.787064 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce7c61-1449-4d61-b643-f22dc7b544aa None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:54.992748 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce7c61-1449-4d61-b643-f22dc7b544aa None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:55.049499 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:55.059578 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff13ba-3cf8-4f09-b243-f79ae7847783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:55.060603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff13ba-3cf8-4f09-b243-f79ae7847783 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:55.061094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff13ba-3cf8-4f09-b243-f79ae7847783 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:55.061572 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff13ba-3cf8-4f09-b243-f79ae7847783 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:55.062538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff13ba-3cf8-4f09-b243-f79ae7847783 None admin] RBAC: Adding query filter params (role.id=92533f29c1d54fe09591e6f2a906ac04, user.id=448f33ef29994926b886d8eb62844a65, scope.system=all) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:55.063080 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff13ba-3cf8-4f09-b243-f79ae7847783 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:55.064848 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff13ba-3cf8-4f09-b243-f79ae7847783 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:55.082558 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 167/334] 10.222.0.67 () {56 vars in 1308 bytes} [Wed Aug 24 08:41:54 2022] GET /identity/v3/role_assignments?user.id=448f33ef29994926b886d8eb62844a65&scope.system=all&role.id=92533f29c1d54fe09591e6f2a906ac04 => generated 510 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:56.108310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d722a-e39b-4b35-825a-5e6f370c494f None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:56.109312 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d722a-e39b-4b35-825a-5e6f370c494f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:56.109799 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d722a-e39b-4b35-825a-5e6f370c494f None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:56.111657 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 168/335] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:56.119024 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174406f-3990-4498-bc62-6eebd861d7f4 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:56.119963 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174406f-3990-4498-bc62-6eebd861d7f4 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:56.120499 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174406f-3990-4498-bc62-6eebd861d7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:56.145803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f174406f-3990-4498-bc62-6eebd861d7f4 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:56.147365 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f174406f-3990-4498-bc62-6eebd861d7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:56.252182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 168/336] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:41:56.578484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d722a-e39b-4b35-825a-5e6f370c494f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:56.776134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:56.799060 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d722a-e39b-4b35-825a-5e6f370c494f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:56.841934 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a1c55-a3e3-4f82-ae35-20cd134df373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:56.842476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a1c55-a3e3-4f82-ae35-20cd134df373 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:56.842755 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a1c55-a3e3-4f82-ae35-20cd134df373 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:56.843047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a1c55-a3e3-4f82-ae35-20cd134df373 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:56.858681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a1c55-a3e3-4f82-ae35-20cd134df373 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:56.863470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-e28a1c55-a3e3-4f82-ae35-20cd134df373 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:56.863470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:56.863470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:56.863470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:56.863470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:56.863470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:56.863470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:56.863470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:56.863470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:56.863470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:56.863470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:56.863470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:56.863470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:56.863470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:56.863470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:56.863470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:56.865291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:56.865291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:56.865291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:56.865291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:56.865291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:56.865291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:56.865291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:56.865291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:56.865291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:56.865291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:56.865291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:56.865291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:56.865291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:56.865291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:56.865291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:56.865291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:56.871291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:56.871291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:56.871291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:56.871291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:56.871291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:56.871291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:56.871291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:56.871291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:56.871291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:56.871291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:56.871291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:56.871291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:41:56.871291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 169/337] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:56.884335 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174406f-3990-4498-bc62-6eebd861d7f4 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:57.053393 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174406f-3990-4498-bc62-6eebd861d7f4 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:57.082859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:57.104905 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e99309-813c-4f0f-9c38-1a83ed9e4f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:57.105876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e99309-813c-4f0f-9c38-1a83ed9e4f9f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:57.106380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e99309-813c-4f0f-9c38-1a83ed9e4f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:57.106879 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e99309-813c-4f0f-9c38-1a83ed9e4f9f None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:57.107774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e99309-813c-4f0f-9c38-1a83ed9e4f9f None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:57.108330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e99309-813c-4f0f-9c38-1a83ed9e4f9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:57.110110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e99309-813c-4f0f-9c38-1a83ed9e4f9f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:57.120511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 169/338] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:57.127149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a1c55-a3e3-4f82-ae35-20cd134df373 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:57.173354 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27133f6-7b80-4f50-8fd6-ca9f9069b00c None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:57.173694 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27133f6-7b80-4f50-8fd6-ca9f9069b00c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:57.173921 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27133f6-7b80-4f50-8fd6-ca9f9069b00c None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:57.176511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 170/339] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:57.181803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af739ee-8cbc-468c-bd24-d6998e1de657 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:57.182178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af739ee-8cbc-468c-bd24-d6998e1de657 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:57.182340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af739ee-8cbc-468c-bd24-d6998e1de657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:57.216156 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7af739ee-8cbc-468c-bd24-d6998e1de657 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:57.218008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7af739ee-8cbc-468c-bd24-d6998e1de657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:57.334769 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 171/340] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:57.338028 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a09d4-62b1-49ea-8c45-c39cac71eb90 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:57.338581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a09d4-62b1-49ea-8c45-c39cac71eb90 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:57.339083 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a09d4-62b1-49ea-8c45-c39cac71eb90 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:57.339703 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a1c55-a3e3-4f82-ae35-20cd134df373 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:57.341899 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 172/341] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:57.348344 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619415e-7d8a-4ab7-baa6-2290ee5f88cf None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:57.349250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619415e-7d8a-4ab7-baa6-2290ee5f88cf None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:57.349819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619415e-7d8a-4ab7-baa6-2290ee5f88cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:57.397403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b619415e-7d8a-4ab7-baa6-2290ee5f88cf None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:57.397792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20a14a-f89e-4b4d-928b-5c64bfa96193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:57.398382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20a14a-f89e-4b4d-928b-5c64bfa96193 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:57.398664 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b619415e-7d8a-4ab7-baa6-2290ee5f88cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:57.398937 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20a14a-f89e-4b4d-928b-5c64bfa96193 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:57.399184 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20a14a-f89e-4b4d-928b-5c64bfa96193 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:57.424894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:57.550865 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20a14a-f89e-4b4d-928b-5c64bfa96193 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:57.552385 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20a14a-f89e-4b4d-928b-5c64bfa96193 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:57.586653 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 173/342] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:57.725049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 170/343] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:41:57.731492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619415e-7d8a-4ab7-baa6-2290ee5f88cf None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:57.807214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:57.871129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619415e-7d8a-4ab7-baa6-2290ee5f88cf None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:57.893092 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20a14a-f89e-4b4d-928b-5c64bfa96193 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:57.919713 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d01d3-13a3-44b5-8084-f5dfdec7ea1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:57.920399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d01d3-13a3-44b5-8084-f5dfdec7ea1c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:57.920674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d01d3-13a3-44b5-8084-f5dfdec7ea1c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:57.920943 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d01d3-13a3-44b5-8084-f5dfdec7ea1c None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:57.922109 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d01d3-13a3-44b5-8084-f5dfdec7ea1c None admin] RBAC: Authorizing `identity:get_project(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:57.923624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d01d3-13a3-44b5-8084-f5dfdec7ea1c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:57.925952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 174/344] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:57 2022] GET /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:57.933144 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d01d3-13a3-44b5-8084-f5dfdec7ea1c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:57.967992 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:58.100488 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20a14a-f89e-4b4d-928b-5c64bfa96193 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:58.122238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d01d3-13a3-44b5-8084-f5dfdec7ea1c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:58.135393 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:58.166174 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7571c4-054d-41b4-9045-aeda20f7bf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:58.166837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7571c4-054d-41b4-9045-aeda20f7bf0e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:58.167110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7571c4-054d-41b4-9045-aeda20f7bf0e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:58.167551 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7571c4-054d-41b4-9045-aeda20f7bf0e None admin] PATH_INFO: `/v3/groups/e96cc682b6194c3480dd474da85b8bac` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:58.175367 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7571c4-054d-41b4-9045-aeda20f7bf0e None admin] RBAC: Authorizing `identity:get_group(group_id=e96cc682b6194c3480dd474da85b8bac)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:58.177383 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7571c4-054d-41b4-9045-aeda20f7bf0e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:58.178939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b23ba-32d1-4211-b7a0-45eaade3f41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:58.179932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b23ba-32d1-4211-b7a0-45eaade3f41a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:58.180272 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b23ba-32d1-4211-b7a0-45eaade3f41a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:58.180548 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b23ba-32d1-4211-b7a0-45eaade3f41a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:58.180940 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 175/345] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 24 08:41:57 2022] GET /identity/v3/groups/e96cc682b6194c3480dd474da85b8bac => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:58.182090 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7571c4-054d-41b4-9045-aeda20f7bf0e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:58.208911 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01b23ba-32d1-4211-b7a0-45eaade3f41a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:58.214479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f01b23ba-32d1-4211-b7a0-45eaade3f41a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:41:58.214479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:58.214479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:58.214479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:58.214479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:58.214479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:58.214479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:58.214479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:58.214479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:58.214479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:58.214479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:58.214479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:58.214479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:58.214479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:58.214479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:58.214479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:58.215581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:58.215581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:58.215581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:58.215581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:58.215581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:58.215581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:58.215581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:58.215581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:58.215581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:58.215581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:58.215581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:58.215581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:58.215581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:58.215581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:58.215581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:58.215581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:58.220764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:58.220764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:58.220764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:58.220764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:58.220764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:58.220764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:58.220764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:58.220764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:58.220764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:58.220764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:58.220764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 08:41:58.220764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:41:58.220764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 171/346] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:41:58.220764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b23ba-32d1-4211-b7a0-45eaade3f41a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:58.314612 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7571c4-054d-41b4-9045-aeda20f7bf0e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:58.359900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118523e-f4fb-475f-a361-821c1dfbbc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:58.360833 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118523e-f4fb-475f-a361-821c1dfbbc4d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:58.361298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118523e-f4fb-475f-a361-821c1dfbbc4d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:58.361729 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118523e-f4fb-475f-a361-821c1dfbbc4d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:58.374651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7118523e-f4fb-475f-a361-821c1dfbbc4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:58.380400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-7118523e-f4fb-475f-a361-821c1dfbbc4d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:41:58.380400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:58.380400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:58.380400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:58.380400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:58.380400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:58.380400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:58.380400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:58.380400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:58.380400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:58.380400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:58.380400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:58.380400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:58.380400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:58.380400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:58.380400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:58.381589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:58.381589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:58.381589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:58.381589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:58.381589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:58.381589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:58.381589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:58.381589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:58.381589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:58.381589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:58.381589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:58.381589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:58.381589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:58.381589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:58.381589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:58.381589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:58.383220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:58.383220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:58.383220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:58.383220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:58.383220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:58.383220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:58.383220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:58.383220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:58.383220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:58.383220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:58.383220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:41:58.383220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:41:58.385646 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 176/347] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:41:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:58.386677 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118523e-f4fb-475f-a361-821c1dfbbc4d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:58.413362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:58.419225 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b23ba-32d1-4211-b7a0-45eaade3f41a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:58.425939 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:58.462192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3e533-54aa-4b23-891f-47032e1263d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:58.462978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3e533-54aa-4b23-891f-47032e1263d5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:58.463336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3e533-54aa-4b23-891f-47032e1263d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:58.463846 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3e533-54aa-4b23-891f-47032e1263d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:58.464848 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3e533-54aa-4b23-891f-47032e1263d5 None admin] RBAC: Adding query filter params (group.id=e96cc682b6194c3480dd474da85b8bac, role.id=92533f29c1d54fe09591e6f2a906ac04, scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:58.465395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3e533-54aa-4b23-891f-47032e1263d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:58.467067 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3e533-54aa-4b23-891f-47032e1263d5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:58.480599 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 172/348] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 24 08:41:58 2022] GET /identity/v3/role_assignments?group.id=e96cc682b6194c3480dd474da85b8bac&scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d&role.id=92533f29c1d54fe09591e6f2a906ac04 => generated 260 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:58.482409 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceed5c-4206-4ff1-be4d-bc26455ea14c None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:58.482919 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceed5c-4206-4ff1-be4d-bc26455ea14c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:58.483382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceed5c-4206-4ff1-be4d-bc26455ea14c None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:58.485265 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 173/349] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:58.492126 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ceed5c-4206-4ff1-be4d-bc26455ea14c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:58.511486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118523e-f4fb-475f-a361-821c1dfbbc4d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:58.557044 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7f6e9-8e1e-4fd8-941d-fa28e18831d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:58.558096 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7f6e9-8e1e-4fd8-941d-fa28e18831d7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:58.558605 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7f6e9-8e1e-4fd8-941d-fa28e18831d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:58.559090 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7f6e9-8e1e-4fd8-941d-fa28e18831d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:58.560062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7f6e9-8e1e-4fd8-941d-fa28e18831d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:58.560630 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7f6e9-8e1e-4fd8-941d-fa28e18831d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:58.562351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7f6e9-8e1e-4fd8-941d-fa28e18831d7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:58.573264 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 177/350] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:58.575109 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5ef2e-87da-4472-8714-d7cd5a6c665a None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:58.575734 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5ef2e-87da-4472-8714-d7cd5a6c665a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:58.580527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5ef2e-87da-4472-8714-d7cd5a6c665a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:58.602775 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-75f5ef2e-87da-4472-8714-d7cd5a6c665a None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:58.604506 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-75f5ef2e-87da-4472-8714-d7cd5a6c665a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:58.685830 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 178/351] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:58.687062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5ef2e-87da-4472-8714-d7cd5a6c665a None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:58.718912 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:58.809929 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5ef2e-87da-4472-8714-d7cd5a6c665a None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:58.821340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ceed5c-4206-4ff1-be4d-bc26455ea14c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:58.852408 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e60583-8ec3-43f6-b77c-3da2a5f84174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:58.853582 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e60583-8ec3-43f6-b77c-3da2a5f84174 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:58.854069 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e60583-8ec3-43f6-b77c-3da2a5f84174 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:58.854542 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e60583-8ec3-43f6-b77c-3da2a5f84174 None admin] PATH_INFO: `/v3/users/e2836a70b2044fa9b6f965349929496e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:58.856015 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e60583-8ec3-43f6-b77c-3da2a5f84174 None admin] RBAC: Authorizing `identity:get_user(user_id=e2836a70b2044fa9b6f965349929496e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:58.857735 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e60583-8ec3-43f6-b77c-3da2a5f84174 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:58.859687 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 179/352] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:41:58 2022] GET /identity/v3/users/e2836a70b2044fa9b6f965349929496e => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:58.865554 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a0ead-ef2a-4b33-85e1-a989df208e68 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:58.866077 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a0ead-ef2a-4b33-85e1-a989df208e68 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:58.866682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a0ead-ef2a-4b33-85e1-a989df208e68 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:58.868312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 180/353] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:58.870577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a0ead-ef2a-4b33-85e1-a989df208e68 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:58.900546 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adea794-2de7-4d21-89b3-d151b426343f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:58.901982 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adea794-2de7-4d21-89b3-d151b426343f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:58.902793 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adea794-2de7-4d21-89b3-d151b426343f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:58.903472 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adea794-2de7-4d21-89b3-d151b426343f None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:58.905351 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adea794-2de7-4d21-89b3-d151b426343f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:58.906191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adea794-2de7-4d21-89b3-d151b426343f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:58.909434 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adea794-2de7-4d21-89b3-d151b426343f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:58.931866 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 174/354] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:41:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:58.934811 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2d42b-66ce-4508-a894-d151c6c73ef7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:58.935591 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2d42b-66ce-4508-a894-d151c6c73ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:58.936405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2d42b-66ce-4508-a894-d151c6c73ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:58.971682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e4f2d42b-66ce-4508-a894-d151c6c73ef7 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:58.974203 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e4f2d42b-66ce-4508-a894-d151c6c73ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:58.990217 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:59.003069 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a0ead-ef2a-4b33-85e1-a989df208e68 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:59.044688 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5f0ff-5757-4d21-bfcc-5562a4d28e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:59.045328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5f0ff-5757-4d21-bfcc-5562a4d28e47 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:59.045611 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5f0ff-5757-4d21-bfcc-5562a4d28e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:59.045887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5f0ff-5757-4d21-bfcc-5562a4d28e47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:59.046643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5f0ff-5757-4d21-bfcc-5562a4d28e47 None admin] RBAC: Adding query filter params (role.id=7f4cf2414d134bd4ac30986436be0095, user.id=e2836a70b2044fa9b6f965349929496e, scope.system=all) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:59.046958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5f0ff-5757-4d21-bfcc-5562a4d28e47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:59.048460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5f0ff-5757-4d21-bfcc-5562a4d28e47 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:59.048631 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:59.062593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 181/355] 10.222.0.67 () {56 vars in 1308 bytes} [Wed Aug 24 08:41:58 2022] GET /identity/v3/role_assignments?user.id=e2836a70b2044fa9b6f965349929496e&scope.system=all&role.id=7f4cf2414d134bd4ac30986436be0095 => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:41:59.069854 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5f0ff-5757-4d21-bfcc-5562a4d28e47 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:59.128914 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 175/356] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:59.188668 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2d42b-66ce-4508-a894-d151c6c73ef7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:59.259887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5f0ff-5757-4d21-bfcc-5562a4d28e47 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:59.339906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f0457-a594-4407-99c9-49093870a088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:59.340566 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f0457-a594-4407-99c9-49093870a088 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:59.341617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f0457-a594-4407-99c9-49093870a088 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:59.341617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f0457-a594-4407-99c9-49093870a088 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:59.377615 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:59.394073 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2d42b-66ce-4508-a894-d151c6c73ef7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:59.441926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a450c-b66b-4839-b9a7-67b27f994ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:59.443060 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a450c-b66b-4839-b9a7-67b27f994ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:59.443549 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a450c-b66b-4839-b9a7-67b27f994ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:59.444023 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a450c-b66b-4839-b9a7-67b27f994ea4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:59.444641 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f0457-a594-4407-99c9-49093870a088 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:59.446168 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f0457-a594-4407-99c9-49093870a088 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:59.458804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a450c-b66b-4839-b9a7-67b27f994ea4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:41:59.466646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-772a450c-b66b-4839-b9a7-67b27f994ea4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:59.466646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:41:59.466646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:41:59.466646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:41:59.466646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:41:59.466646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:41:59.466646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:41:59.466646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:41:59.466646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:41:59.466646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:41:59.466646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:41:59.466646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:41:59.466646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:41:59.466646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:41:59.466646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:41:59.466646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:41:59.467607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:41:59.467607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:41:59.467607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:41:59.467607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:41:59.467607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:41:59.467607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:41:59.467607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:41:59.467607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:41:59.467607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:41:59.467607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:41:59.467607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:41:59.467607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:41:59.467607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:41:59.467607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:41:59.467607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:41:59.467607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:41:59.468449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:41:59.468449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:41:59.468449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:41:59.468449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:41:59.468449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:41:59.468449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:41:59.468449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:41:59.468449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:41:59.468449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:41:59.468449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:41:59.468449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:41:59.468449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:41:59.471353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 176/357] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:41:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:41:59.473152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d79f70-280b-4107-97b2-142bcc610146 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:59.473791 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d79f70-280b-4107-97b2-142bcc610146 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:59.474269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d79f70-280b-4107-97b2-142bcc610146 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:59.476649 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 177/358] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:41:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:41:59.477740 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d79f70-280b-4107-97b2-142bcc610146 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:59.505478 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:59.567056 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 182/359] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:41:59.572619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8d292-b927-4c56-ad4a-efa8ca6e878e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:59.572993 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8d292-b927-4c56-ad4a-efa8ca6e878e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:59.573343 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8d292-b927-4c56-ad4a-efa8ca6e878e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:59.601104 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-61e8d292-b927-4c56-ad4a-efa8ca6e878e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:41:59.602786 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-61e8d292-b927-4c56-ad4a-efa8ca6e878e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:41:59.603775 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d79f70-280b-4107-97b2-142bcc610146 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:59.643914 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d2641-82a8-476f-8d78-fcad6199d4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:59.644963 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d2641-82a8-476f-8d78-fcad6199d4bb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:59.645461 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d2641-82a8-476f-8d78-fcad6199d4bb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:59.645937 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d2641-82a8-476f-8d78-fcad6199d4bb None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:59.648045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d2641-82a8-476f-8d78-fcad6199d4bb None admin] RBAC: Adding query filter params (name=member) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:41:59.648665 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d2641-82a8-476f-8d78-fcad6199d4bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:59.650555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d2641-82a8-476f-8d78-fcad6199d4bb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:59.658584 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:59.662415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 178/360] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:41:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:59.664337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d2641-82a8-476f-8d78-fcad6199d4bb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:59.682296 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 183/361] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:41:59.683180 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8d292-b927-4c56-ad4a-efa8ca6e878e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:59.795228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:41:59.836852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d2641-82a8-476f-8d78-fcad6199d4bb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:59.904168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7a730-c75a-47b5-9667-86ce706071f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:59.905171 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7a730-c75a-47b5-9667-86ce706071f7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:59.905635 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7a730-c75a-47b5-9667-86ce706071f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:59.905739 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8d292-b927-4c56-ad4a-efa8ca6e878e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:41:59.906082 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7a730-c75a-47b5-9667-86ce706071f7 None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:41:59.908353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7a730-c75a-47b5-9667-86ce706071f7 None admin] RBAC: Authorizing `identity:get_project(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:41:59.911112 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7a730-c75a-47b5-9667-86ce706071f7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:41:59.914490 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 179/362] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:41:59 2022] GET /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:41:59.915913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7a730-c75a-47b5-9667-86ce706071f7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:41:59.947241 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e94a1a-8893-4682-b8c1-fc43841efc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:41:59.947946 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e94a1a-8893-4682-b8c1-fc43841efc19 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:41:59.948255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e94a1a-8893-4682-b8c1-fc43841efc19 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:41:59.948527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e94a1a-8893-4682-b8c1-fc43841efc19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:00.005774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:00.029615 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e94a1a-8893-4682-b8c1-fc43841efc19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:00.031157 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e94a1a-8893-4682-b8c1-fc43841efc19 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:00.108111 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7a730-c75a-47b5-9667-86ce706071f7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:00.170592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e542236-1159-476a-92a1-38840008bf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:00.171262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e542236-1159-476a-92a1-38840008bf07 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:00.171545 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e542236-1159-476a-92a1-38840008bf07 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:00.171821 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e542236-1159-476a-92a1-38840008bf07 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:00.190129 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e542236-1159-476a-92a1-38840008bf07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:00.197931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-9e542236-1159-476a-92a1-38840008bf07 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:42:00.197931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:00.197931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:00.197931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:00.197931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:00.197931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:00.197931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:00.197931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:00.197931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:00.197931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:00.197931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:00.197931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:00.197931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:00.197931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:00.197931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:00.197931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:00.198883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:00.198883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:00.198883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:00.198883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:00.198883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:00.198883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:00.198883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:00.198883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:00.198883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:00.198883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:00.198883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:00.198883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:00.198883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:00.198883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:00.198883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:00.198883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:00.201186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:00.201186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:00.201186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:00.201186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:00.201186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:00.201186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:00.201186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:00.201186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:00.201186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:00.201186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:00.201186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:42:00.201186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:00.201186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 180/363] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:41:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:00.201186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e542236-1159-476a-92a1-38840008bf07 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:00.213130 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:00.225635 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 184/364] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:41:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:42:00.227169 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e94a1a-8893-4682-b8c1-fc43841efc19 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:00.361098 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e542236-1159-476a-92a1-38840008bf07 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:00.380882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:00.419806 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e94a1a-8893-4682-b8c1-fc43841efc19 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:00.432132 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d501f3c-59d1-4170-bc34-cc7927a4877a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:00.432132 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d501f3c-59d1-4170-bc34-cc7927a4877a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:00.432477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d501f3c-59d1-4170-bc34-cc7927a4877a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:00.432642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d501f3c-59d1-4170-bc34-cc7927a4877a None admin] PATH_INFO: `/v3/groups/e96cc682b6194c3480dd474da85b8bac` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:00.434667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d501f3c-59d1-4170-bc34-cc7927a4877a None admin] RBAC: Authorizing `identity:get_group(group_id=e96cc682b6194c3480dd474da85b8bac)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:00.436183 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d501f3c-59d1-4170-bc34-cc7927a4877a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:00.438117 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 181/365] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 24 08:42:00 2022] GET /identity/v3/groups/e96cc682b6194c3480dd474da85b8bac => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:00.438640 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d501f3c-59d1-4170-bc34-cc7927a4877a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:00.491443 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed23f1-0422-4f86-9717-cf4e287969bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:00.492550 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed23f1-0422-4f86-9717-cf4e287969bf None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:00.493021 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed23f1-0422-4f86-9717-cf4e287969bf None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:00.493495 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed23f1-0422-4f86-9717-cf4e287969bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:00.512807 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed23f1-0422-4f86-9717-cf4e287969bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:00.518082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-eeed23f1-0422-4f86-9717-cf4e287969bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:42:00.518082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:00.518082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:00.518082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:00.518082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:00.518082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:00.518082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:00.518082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:00.518082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:00.518082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:00.518082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:00.518082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:00.518082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:00.518082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:00.518082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:00.518082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:00.519008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:00.519008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:00.519008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:00.519008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:00.519008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:00.519008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:00.519008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:00.519008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:00.519008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:00.519008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:00.519008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:00.519008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:00.519008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:00.519008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:00.519008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:00.519008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:00.519803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:00.519803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:00.519803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:00.519803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:00.519803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:00.519803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:00.519803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:00.519803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:00.519803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:00.519803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:00.519803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:42:00.519803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:00.522608 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 185/366] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:42:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:00.523784 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed23f1-0422-4f86-9717-cf4e287969bf None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:00.581370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:00.626665 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d501f3c-59d1-4170-bc34-cc7927a4877a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:00.668271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc323cb1-f84d-4605-b2cb-eb2bd0dcb823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:00.668271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc323cb1-f84d-4605-b2cb-eb2bd0dcb823 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:00.668702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc323cb1-f84d-4605-b2cb-eb2bd0dcb823 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:00.668702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc323cb1-f84d-4605-b2cb-eb2bd0dcb823 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:00.669257 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc323cb1-f84d-4605-b2cb-eb2bd0dcb823 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:00.669553 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc323cb1-f84d-4605-b2cb-eb2bd0dcb823 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:00.670950 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc323cb1-f84d-4605-b2cb-eb2bd0dcb823 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:00.680908 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 182/367] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:00.681766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc323cb1-f84d-4605-b2cb-eb2bd0dcb823 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:00.732210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed23f1-0422-4f86-9717-cf4e287969bf None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:00.813091 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff92ce-e1ee-4a6a-8023-2edf0d01260f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:00.815512 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff92ce-e1ee-4a6a-8023-2edf0d01260f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:00.816005 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff92ce-e1ee-4a6a-8023-2edf0d01260f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:00.816862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff92ce-e1ee-4a6a-8023-2edf0d01260f None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:00.819038 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff92ce-e1ee-4a6a-8023-2edf0d01260f None admin] RBAC: Authorizing `identity:get_project(project_id=9381518952174868a5bc99813df0342e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:00.821977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff92ce-e1ee-4a6a-8023-2edf0d01260f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:00.827573 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 186/368] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:42:00 2022] GET /identity/v3/projects/9381518952174868a5bc99813df0342e => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:00.828920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff92ce-e1ee-4a6a-8023-2edf0d01260f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:00.855630 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:00.860391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc323cb1-f84d-4605-b2cb-eb2bd0dcb823 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:00.931576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426fc29-4d9d-4e2c-ad5d-362cd8145615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:00.933521 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426fc29-4d9d-4e2c-ad5d-362cd8145615 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:00.934260 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426fc29-4d9d-4e2c-ad5d-362cd8145615 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:00.934708 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426fc29-4d9d-4e2c-ad5d-362cd8145615 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:00.937663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426fc29-4d9d-4e2c-ad5d-362cd8145615 None admin] RBAC: Adding query filter params (group.id=e96cc682b6194c3480dd474da85b8bac, role.id=5915b1954b514151ab17ff5d531036e4, scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:00.937663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426fc29-4d9d-4e2c-ad5d-362cd8145615 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:00.939838 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426fc29-4d9d-4e2c-ad5d-362cd8145615 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:00.948461 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:00.955393 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 183/369] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 24 08:42:00 2022] GET /identity/v3/role_assignments?group.id=e96cc682b6194c3480dd474da85b8bac&scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d&role.id=5915b1954b514151ab17ff5d531036e4 => generated 260 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:00.957407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426fc29-4d9d-4e2c-ad5d-362cd8145615 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:01.073572 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff92ce-e1ee-4a6a-8023-2edf0d01260f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:01.128152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a54f50-897f-4ef9-9083-dc1909d450a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:01.128815 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a54f50-897f-4ef9-9083-dc1909d450a7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:01.129093 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a54f50-897f-4ef9-9083-dc1909d450a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:01.129370 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a54f50-897f-4ef9-9083-dc1909d450a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:01.130069 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a54f50-897f-4ef9-9083-dc1909d450a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:01.130373 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a54f50-897f-4ef9-9083-dc1909d450a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:01.132949 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a54f50-897f-4ef9-9083-dc1909d450a7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:01.144400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 187/370] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:42:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:01.145312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a54f50-897f-4ef9-9083-dc1909d450a7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:01.154666 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426fc29-4d9d-4e2c-ad5d-362cd8145615 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:01.198505 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969ffee-c83f-4061-b75e-15b637ff1a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:01.199145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969ffee-c83f-4061-b75e-15b637ff1a43 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:01.199406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969ffee-c83f-4061-b75e-15b637ff1a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:01.199623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969ffee-c83f-4061-b75e-15b637ff1a43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:01.290843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:01.294138 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969ffee-c83f-4061-b75e-15b637ff1a43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:01.296208 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969ffee-c83f-4061-b75e-15b637ff1a43 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:01.298491 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:01.367199 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a54f50-897f-4ef9-9083-dc1909d450a7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:01.413682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 184/371] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:42:01.419951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969ffee-c83f-4061-b75e-15b637ff1a43 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:01.422875 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5328eb-9128-46cf-b100-23d23a3ba6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:01.423685 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5328eb-9128-46cf-b100-23d23a3ba6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:01.424002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5328eb-9128-46cf-b100-23d23a3ba6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:01.425449 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5328eb-9128-46cf-b100-23d23a3ba6d9 None admin] PATH_INFO: `/v3/groups/e96cc682b6194c3480dd474da85b8bac` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:01.427729 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5328eb-9128-46cf-b100-23d23a3ba6d9 None admin] RBAC: Authorizing `identity:get_group(group_id=e96cc682b6194c3480dd474da85b8bac)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:01.429872 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5328eb-9128-46cf-b100-23d23a3ba6d9 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:01.436147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 188/372] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 24 08:42:01 2022] GET /identity/v3/groups/e96cc682b6194c3480dd474da85b8bac => generated 226 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:01.436147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5328eb-9128-46cf-b100-23d23a3ba6d9 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:01.565378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:01.570801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969ffee-c83f-4061-b75e-15b637ff1a43 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:01.586836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5328eb-9128-46cf-b100-23d23a3ba6d9 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:01.634360 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d989a9-a795-4722-90ae-964f4d458fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:01.635027 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d989a9-a795-4722-90ae-964f4d458fab None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:01.635310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d989a9-a795-4722-90ae-964f4d458fab None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:01.635585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d989a9-a795-4722-90ae-964f4d458fab None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:01.637189 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d989a9-a795-4722-90ae-964f4d458fab None admin] RBAC: Authorizing `identity:get_project(project_id=9381518952174868a5bc99813df0342e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:01.638666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d989a9-a795-4722-90ae-964f4d458fab None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:01.640737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 189/373] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:42:01 2022] GET /identity/v3/projects/9381518952174868a5bc99813df0342e => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:01.641537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d989a9-a795-4722-90ae-964f4d458fab None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:01.643547 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d42e49-dbc0-405c-8df8-77bb4b4a102d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:01.645063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d42e49-dbc0-405c-8df8-77bb4b4a102d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:01.645832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d42e49-dbc0-405c-8df8-77bb4b4a102d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:01.646730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d42e49-dbc0-405c-8df8-77bb4b4a102d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:01.655282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:01.764307 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d42e49-dbc0-405c-8df8-77bb4b4a102d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:01.766106 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d42e49-dbc0-405c-8df8-77bb4b4a102d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:01.838512 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d989a9-a795-4722-90ae-964f4d458fab None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:01.888854 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33de48-df96-4b4f-834f-ee15b9d0698c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:01.890060 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33de48-df96-4b4f-834f-ee15b9d0698c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:01.890512 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33de48-df96-4b4f-834f-ee15b9d0698c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:01.891455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33de48-df96-4b4f-834f-ee15b9d0698c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:01.892709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33de48-df96-4b4f-834f-ee15b9d0698c None admin] RBAC: Adding query filter params (group.id=e96cc682b6194c3480dd474da85b8bac, role.id=92533f29c1d54fe09591e6f2a906ac04, scope.project.id=9381518952174868a5bc99813df0342e) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:01.893186 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33de48-df96-4b4f-834f-ee15b9d0698c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:01.895702 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33de48-df96-4b4f-834f-ee15b9d0698c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:01.900323 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:01.904873 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 190/374] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 24 08:42:01 2022] GET /identity/v3/role_assignments?group.id=e96cc682b6194c3480dd474da85b8bac&scope.project.id=9381518952174868a5bc99813df0342e&role.id=92533f29c1d54fe09591e6f2a906ac04 => generated 260 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:01.906189 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b94326-a5eb-4691-8063-95b2bc8c3a1c None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:01.906476 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b94326-a5eb-4691-8063-95b2bc8c3a1c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:01.906751 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b94326-a5eb-4691-8063-95b2bc8c3a1c None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:01.908253 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 191/375] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:01.908832 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b94326-a5eb-4691-8063-95b2bc8c3a1c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:01.915747 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 185/376] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:42:01.920692 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cefb1-a521-4242-bcff-92c7864907a5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:01.923791 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cefb1-a521-4242-bcff-92c7864907a5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:01.924325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cefb1-a521-4242-bcff-92c7864907a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:01.958643 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9c3cefb1-a521-4242-bcff-92c7864907a5 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:01.960645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9c3cefb1-a521-4242-bcff-92c7864907a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:02.079177 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:02.107627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 186/377] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:02.109733 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3cefb1-a521-4242-bcff-92c7864907a5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:02.175678 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b94326-a5eb-4691-8063-95b2bc8c3a1c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:02.247580 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109445de-7852-4343-ae73-18b44c82c3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:02.248289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109445de-7852-4343-ae73-18b44c82c3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:02.248574 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109445de-7852-4343-ae73-18b44c82c3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:02.248848 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109445de-7852-4343-ae73-18b44c82c3f3 None admin] PATH_INFO: `/v3/groups/e96cc682b6194c3480dd474da85b8bac` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:02.250786 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109445de-7852-4343-ae73-18b44c82c3f3 None admin] RBAC: Authorizing `identity:get_group(group_id=e96cc682b6194c3480dd474da85b8bac)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:02.251927 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109445de-7852-4343-ae73-18b44c82c3f3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:02.256294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 192/378] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 24 08:42:01 2022] GET /identity/v3/groups/e96cc682b6194c3480dd474da85b8bac => generated 226 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:02.262434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109445de-7852-4343-ae73-18b44c82c3f3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:02.419985 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3cefb1-a521-4242-bcff-92c7864907a5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:02.519350 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:02.539369 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519ae30-d9a8-4382-982b-f24f404ffbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:02.543114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519ae30-d9a8-4382-982b-f24f404ffbd2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:02.544129 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519ae30-d9a8-4382-982b-f24f404ffbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:02.549442 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519ae30-d9a8-4382-982b-f24f404ffbd2 None admin] PATH_INFO: `/v3/projects/456ae4386c984b51b020a86192cad430` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:02.552611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519ae30-d9a8-4382-982b-f24f404ffbd2 None admin] RBAC: Authorizing `identity:get_project(project_id=456ae4386c984b51b020a86192cad430)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:02.556017 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519ae30-d9a8-4382-982b-f24f404ffbd2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:02.563014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 187/379] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:42:02 2022] GET /identity/v3/projects/456ae4386c984b51b020a86192cad430 => generated 344 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:02.573649 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519ae30-d9a8-4382-982b-f24f404ffbd2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:02.600478 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109445de-7852-4343-ae73-18b44c82c3f3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:02.690195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd8ec6-2352-4a63-95e7-bea14185957c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:02.691278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd8ec6-2352-4a63-95e7-bea14185957c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:02.691777 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd8ec6-2352-4a63-95e7-bea14185957c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:02.692283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd8ec6-2352-4a63-95e7-bea14185957c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:02.693850 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd8ec6-2352-4a63-95e7-bea14185957c None admin] RBAC: Adding query filter params (group.id=e96cc682b6194c3480dd474da85b8bac, role.id=5915b1954b514151ab17ff5d531036e4, scope.project.id=9381518952174868a5bc99813df0342e) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:02.694901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd8ec6-2352-4a63-95e7-bea14185957c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:02.697864 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd8ec6-2352-4a63-95e7-bea14185957c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:02.713901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 193/380] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 24 08:42:02 2022] GET /identity/v3/role_assignments?group.id=e96cc682b6194c3480dd474da85b8bac&scope.project.id=9381518952174868a5bc99813df0342e&role.id=5915b1954b514151ab17ff5d531036e4 => generated 260 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:02.714839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd8ec6-2352-4a63-95e7-bea14185957c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:02.740747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:02.880409 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519ae30-d9a8-4382-982b-f24f404ffbd2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:02.906614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd8ec6-2352-4a63-95e7-bea14185957c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:02.952176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bf05e-1a04-4e50-9d08-3a6cc28e8931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:02.953107 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bf05e-1a04-4e50-9d08-3a6cc28e8931 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:02.953558 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bf05e-1a04-4e50-9d08-3a6cc28e8931 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:02.953981 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bf05e-1a04-4e50-9d08-3a6cc28e8931 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:02.967212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bf05e-1a04-4e50-9d08-3a6cc28e8931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:02.972958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-e30bf05e-1a04-4e50-9d08-3a6cc28e8931 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:42:02.972958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:02.972958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:02.972958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:02.972958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:02.972958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:02.972958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:02.972958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:02.972958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:02.972958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:02.972958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:02.972958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:02.972958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:02.972958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:02.972958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:02.972958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:02.974039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:02.974039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:02.974039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:02.974039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:02.974039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:02.974039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:02.974039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:02.974039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:02.974039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:02.974039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:02.974039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:02.974039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:02.974039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:02.974039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:02.974039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:02.974039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:02.974891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:02.974891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:02.974891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:02.974891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:02.974891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:02.974891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:02.974891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:02.974891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:02.974891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:02.974891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:02.974891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:42:02.974891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:02.977309 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 194/381] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:42:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:02.986466 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bf05e-1a04-4e50-9d08-3a6cc28e8931 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:02.989346 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24e9e0-15b4-4c09-a029-9a7a9e1d8343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:02.990383 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24e9e0-15b4-4c09-a029-9a7a9e1d8343 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:02.990883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24e9e0-15b4-4c09-a029-9a7a9e1d8343 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:02.991775 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24e9e0-15b4-4c09-a029-9a7a9e1d8343 None admin] PATH_INFO: `/v3/groups/612063f7d74f41b79d34ab45962a78fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:03.002891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24e9e0-15b4-4c09-a029-9a7a9e1d8343 None admin] RBAC: Authorizing `identity:get_group(group_id=612063f7d74f41b79d34ab45962a78fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:03.005584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24e9e0-15b4-4c09-a029-9a7a9e1d8343 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:03.010715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 188/382] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 24 08:42:02 2022] GET /identity/v3/groups/612063f7d74f41b79d34ab45962a78fd => generated 229 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:03.020581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24e9e0-15b4-4c09-a029-9a7a9e1d8343 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:03.104756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:03.152486 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24e9e0-15b4-4c09-a029-9a7a9e1d8343 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:03.195576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8f506-f3e4-4d94-aec5-9a982f826f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:03.196663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8f506-f3e4-4d94-aec5-9a982f826f61 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:03.197172 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8f506-f3e4-4d94-aec5-9a982f826f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:03.197655 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8f506-f3e4-4d94-aec5-9a982f826f61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:03.198623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8f506-f3e4-4d94-aec5-9a982f826f61 None admin] RBAC: Adding query filter params (group.id=612063f7d74f41b79d34ab45962a78fd, role.id=b13ffa72344c4856901b1ba003e273fd, scope.project.id=456ae4386c984b51b020a86192cad430) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:03.199161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8f506-f3e4-4d94-aec5-9a982f826f61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:03.200937 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8f506-f3e4-4d94-aec5-9a982f826f61 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:03.201960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:03.208968 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 189/383] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 24 08:42:03 2022] GET /identity/v3/role_assignments?group.id=612063f7d74f41b79d34ab45962a78fd&scope.project.id=456ae4386c984b51b020a86192cad430&role.id=b13ffa72344c4856901b1ba003e273fd => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:03.315178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bf05e-1a04-4e50-9d08-3a6cc28e8931 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:03.393672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01d07e-7052-4b30-b8d5-70375628ba9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:03.394632 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01d07e-7052-4b30-b8d5-70375628ba9e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:03.395045 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01d07e-7052-4b30-b8d5-70375628ba9e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:03.395446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01d07e-7052-4b30-b8d5-70375628ba9e None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:03.396587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01d07e-7052-4b30-b8d5-70375628ba9e None admin] RBAC: Adding query filter params (name=member) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:03.397073 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01d07e-7052-4b30-b8d5-70375628ba9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:03.399484 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01d07e-7052-4b30-b8d5-70375628ba9e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:03.415319 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 195/384] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:42:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:03.424289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8f506-f3e4-4d94-aec5-9a982f826f61 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:03.671272 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8f506-f3e4-4d94-aec5-9a982f826f61 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:03.729182 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:03.752583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04a8ca-b09c-4f0a-bfbf-1028d1fdea8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:03.753246 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04a8ca-b09c-4f0a-bfbf-1028d1fdea8a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:03.753529 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04a8ca-b09c-4f0a-bfbf-1028d1fdea8a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:03.753798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04a8ca-b09c-4f0a-bfbf-1028d1fdea8a None admin] PATH_INFO: `/v3/groups/e96cc682b6194c3480dd474da85b8bac` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:03.757016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f04a8ca-b09c-4f0a-bfbf-1028d1fdea8a None admin] RBAC: Authorizing `identity:get_group(group_id=e96cc682b6194c3480dd474da85b8bac)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:03.758698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f04a8ca-b09c-4f0a-bfbf-1028d1fdea8a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:03.760801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 190/385] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 24 08:42:03 2022] GET /identity/v3/groups/e96cc682b6194c3480dd474da85b8bac => generated 226 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:03.766106 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01d07e-7052-4b30-b8d5-70375628ba9e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:03.921620 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01d07e-7052-4b30-b8d5-70375628ba9e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:03.964692 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:03.969536 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee04ec5-9f3e-46a6-ae8c-f03d8bc5fd24 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:03.969902 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee04ec5-9f3e-46a6-ae8c-f03d8bc5fd24 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:03.970103 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee04ec5-9f3e-46a6-ae8c-f03d8bc5fd24 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:03.971836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 191/386] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:03.979030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199308d3-4f37-4348-adb6-68a29bccdfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:03.979700 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199308d3-4f37-4348-adb6-68a29bccdfcf None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:03.979977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199308d3-4f37-4348-adb6-68a29bccdfcf None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:03.980369 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199308d3-4f37-4348-adb6-68a29bccdfcf None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:03.981520 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c7f8f-a952-4d9b-89b2-7eab8a7a5354 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:03.982006 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c7f8f-a952-4d9b-89b2-7eab8a7a5354 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:03.982495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c7f8f-a952-4d9b-89b2-7eab8a7a5354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:03.985012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199308d3-4f37-4348-adb6-68a29bccdfcf None admin] RBAC: Authorizing `identity:get_project(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:03.986508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199308d3-4f37-4348-adb6-68a29bccdfcf None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:03.988875 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 196/387] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:42:03 2022] GET /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:03.995234 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199308d3-4f37-4348-adb6-68a29bccdfcf None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:04.025249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e02c7f8f-a952-4d9b-89b2-7eab8a7a5354 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:04.026476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e02c7f8f-a952-4d9b-89b2-7eab8a7a5354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:04.170427 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 192/388] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:42:04.180426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199308d3-4f37-4348-adb6-68a29bccdfcf None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:04.240972 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec622a5-775f-4a81-b3bb-300146471bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:04.241742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec622a5-775f-4a81-b3bb-300146471bcd None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:04.242021 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec622a5-775f-4a81-b3bb-300146471bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:04.242298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec622a5-775f-4a81-b3bb-300146471bcd None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d/groups/e96cc682b6194c3480dd474da85b8bac/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:04.244655 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec622a5-775f-4a81-b3bb-300146471bcd None admin] RBAC: Authorizing `identity:create_grant(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d, group_id=e96cc682b6194c3480dd474da85b8bac, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:04.246148 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec622a5-775f-4a81-b3bb-300146471bcd None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:04.272762 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 197/389] 10.222.0.67 () {58 vars in 1442 bytes} [Wed Aug 24 08:42:03 2022] PUT /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d/groups/e96cc682b6194c3480dd474da85b8bac/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:42:04.793627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c7f8f-a952-4d9b-89b2-7eab8a7a5354 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:04.858372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332de2f-e5c8-4ff0-b6c7-cb6a1dedba52 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:04.859142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332de2f-e5c8-4ff0-b6c7-cb6a1dedba52 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:04.859142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332de2f-e5c8-4ff0-b6c7-cb6a1dedba52 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:04.860643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 198/390] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:42:04.886130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb047aad-0aec-4d63-92b1-e64e35290723 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:04.886544 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb047aad-0aec-4d63-92b1-e64e35290723 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:04.886783 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb047aad-0aec-4d63-92b1-e64e35290723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:04.920121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-eb047aad-0aec-4d63-92b1-e64e35290723 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:04.921315 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-eb047aad-0aec-4d63-92b1-e64e35290723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:05.003436 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c7f8f-a952-4d9b-89b2-7eab8a7a5354 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:05.035216 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:05.048688 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6d8f3-cb43-4451-8c3e-ed48821cc1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:05.049334 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6d8f3-cb43-4451-8c3e-ed48821cc1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:05.049610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6d8f3-cb43-4451-8c3e-ed48821cc1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:05.050134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6d8f3-cb43-4451-8c3e-ed48821cc1e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:05.062681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce6d8f3-cb43-4451-8c3e-ed48821cc1e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:05.069642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-7ce6d8f3-cb43-4451-8c3e-ed48821cc1e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:42:05.069642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:05.069642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:05.069642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:05.069642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:05.069642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:05.069642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:05.069642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:05.069642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:05.069642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:05.069642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:05.069642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:05.069642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:05.069642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:05.069642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:05.069642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:05.070805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:05.070805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:05.070805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:05.070805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:05.070805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:05.070805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:05.070805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:05.070805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:05.070805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:05.070805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:05.070805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:05.070805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:05.070805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:05.070805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:05.070805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:05.070805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:05.071615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:05.071615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:05.071615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:05.071615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:05.071615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:05.071615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:05.071615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:05.071615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:05.071615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:05.071615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:05.071615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:42:05.071615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:05.071615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 193/391] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:42:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:05.072460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:05.082370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6d8f3-cb43-4451-8c3e-ed48821cc1e9 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:05.111460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 199/392] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:05.290987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6d8f3-cb43-4451-8c3e-ed48821cc1e9 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:05.353592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e560ade-262a-4d0e-902c-58f2b9394232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:05.354234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e560ade-262a-4d0e-902c-58f2b9394232 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:05.354514 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e560ade-262a-4d0e-902c-58f2b9394232 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:05.354775 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e560ade-262a-4d0e-902c-58f2b9394232 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:05.355485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e560ade-262a-4d0e-902c-58f2b9394232 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:05.355785 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e560ade-262a-4d0e-902c-58f2b9394232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:05.357261 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e560ade-262a-4d0e-902c-58f2b9394232 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:05.371153 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 194/393] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:05.379348 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb047aad-0aec-4d63-92b1-e64e35290723 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:05.431470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e560ade-262a-4d0e-902c-58f2b9394232 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:05.550332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:05.587570 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb047aad-0aec-4d63-92b1-e64e35290723 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:05.632082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894c7d4-6a94-41db-8e43-64d46adbeee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:05.633717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894c7d4-6a94-41db-8e43-64d46adbeee9 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:05.634508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894c7d4-6a94-41db-8e43-64d46adbeee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:05.635105 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894c7d4-6a94-41db-8e43-64d46adbeee9 None admin] PATH_INFO: `/v3/groups/e96cc682b6194c3480dd474da85b8bac` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:05.636752 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894c7d4-6a94-41db-8e43-64d46adbeee9 None admin] RBAC: Authorizing `identity:get_group(group_id=e96cc682b6194c3480dd474da85b8bac)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:05.639079 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894c7d4-6a94-41db-8e43-64d46adbeee9 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:05.642757 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 200/394] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 24 08:42:05 2022] GET /identity/v3/groups/e96cc682b6194c3480dd474da85b8bac => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:05.648703 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894c7d4-6a94-41db-8e43-64d46adbeee9 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:05.714125 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e560ade-262a-4d0e-902c-58f2b9394232 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:05.784870 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:05.794191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba70773b-560b-4dbf-929e-9f316d3a184f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:05.795032 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70773b-560b-4dbf-929e-9f316d3a184f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:05.795175 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70773b-560b-4dbf-929e-9f316d3a184f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:05.795564 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70773b-560b-4dbf-929e-9f316d3a184f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:05.811953 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba70773b-560b-4dbf-929e-9f316d3a184f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:05.817913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ba70773b-560b-4dbf-929e-9f316d3a184f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:42:05.817913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:05.817913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:05.817913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:05.817913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:05.817913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:05.817913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:05.817913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:05.817913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:05.817913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:05.817913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:05.817913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:05.817913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:05.817913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:05.817913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:05.817913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:05.820444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:05.820444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:05.820444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:05.820444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:05.820444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:05.820444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:05.820444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:05.820444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:05.820444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:05.820444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:05.820444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:05.820444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:05.820444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:05.820444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:05.820444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:05.820444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:05.822034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:05.822034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:05.822034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:05.822034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:05.822034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:05.822034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:05.822034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:05.822034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:05.822034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:05.822034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:05.822034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:42:05.822034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:05.822034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 195/395] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:42:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:42:05.822034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4e12f-6fbb-4eed-81ae-94c269bd1ae9 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:05.824686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4e12f-6fbb-4eed-81ae-94c269bd1ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:05.824686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4e12f-6fbb-4eed-81ae-94c269bd1ae9 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:05.824686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 196/396] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:05.826307 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4e12f-6fbb-4eed-81ae-94c269bd1ae9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:05.906563 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894c7d4-6a94-41db-8e43-64d46adbeee9 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:05.968408 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c41d2-7d38-4130-912b-3cd3e0bdb387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:05.969276 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c41d2-7d38-4130-912b-3cd3e0bdb387 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:05.969654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c41d2-7d38-4130-912b-3cd3e0bdb387 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:05.970049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c41d2-7d38-4130-912b-3cd3e0bdb387 None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:05.971351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c41d2-7d38-4130-912b-3cd3e0bdb387 None admin] RBAC: Authorizing `identity:get_project(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:05.972989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c41d2-7d38-4130-912b-3cd3e0bdb387 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:05.975692 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 201/397] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:42:05 2022] GET /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d => generated 298 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:05.978153 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d8e23-8b2b-4ea1-8ae4-b54cb3a7a590 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:05.978549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d8e23-8b2b-4ea1-8ae4-b54cb3a7a590 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:05.978941 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d8e23-8b2b-4ea1-8ae4-b54cb3a7a590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:06.004869 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-c84d8e23-8b2b-4ea1-8ae4-b54cb3a7a590 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:06.006657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c84d8e23-8b2b-4ea1-8ae4-b54cb3a7a590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:06.028742 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4e12f-6fbb-4eed-81ae-94c269bd1ae9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:06.043378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:06.087066 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7f28a-1886-44a5-8788-6cdd4bb37557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:06.087958 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7f28a-1886-44a5-8788-6cdd4bb37557 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:06.088532 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7f28a-1886-44a5-8788-6cdd4bb37557 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:06.089116 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7f28a-1886-44a5-8788-6cdd4bb37557 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:06.090105 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7f28a-1886-44a5-8788-6cdd4bb37557 None admin] RBAC: Adding query filter params (name=member) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:06.090878 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7f28a-1886-44a5-8788-6cdd4bb37557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:06.092895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7f28a-1886-44a5-8788-6cdd4bb37557 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:06.104148 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 197/398] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:42:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:06.105794 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7f28a-1886-44a5-8788-6cdd4bb37557 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:06.144208 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 202/399] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:06.146323 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d8e23-8b2b-4ea1-8ae4-b54cb3a7a590 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:06.280187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:06.339958 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7f28a-1886-44a5-8788-6cdd4bb37557 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:06.364785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d8e23-8b2b-4ea1-8ae4-b54cb3a7a590 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:06.392626 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4eb20-7a67-4b2b-b254-f9744efb444f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:06.393313 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4eb20-7a67-4b2b-b254-f9744efb444f None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:06.393597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4eb20-7a67-4b2b-b254-f9744efb444f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:06.393872 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4eb20-7a67-4b2b-b254-f9744efb444f None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d/groups/e96cc682b6194c3480dd474da85b8bac/roles/5915b1954b514151ab17ff5d531036e4` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:06.396598 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4eb20-7a67-4b2b-b254-f9744efb444f None admin] RBAC: Authorizing `identity:create_grant(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d, group_id=e96cc682b6194c3480dd474da85b8bac, role_id=5915b1954b514151ab17ff5d531036e4)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:06.398954 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4eb20-7a67-4b2b-b254-f9744efb444f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:06.418042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 198/400] 10.222.0.67 () {58 vars in 1442 bytes} [Wed Aug 24 08:42:06 2022] PUT /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d/groups/e96cc682b6194c3480dd474da85b8bac/roles/5915b1954b514151ab17ff5d531036e4 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:42:06.420545 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe39d85-c056-4ff0-a4e9-437268ad90ba None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:06.420892 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe39d85-c056-4ff0-a4e9-437268ad90ba None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:06.421173 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe39d85-c056-4ff0-a4e9-437268ad90ba None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:06.428974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 199/401] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:06.430350 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe77d28-ee91-4034-bc28-03083f443dfe None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:06.430648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe77d28-ee91-4034-bc28-03083f443dfe None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:06.430912 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe77d28-ee91-4034-bc28-03083f443dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:06.440149 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a746ec-5193-4d59-bc3c-5191a79f3bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:06.441844 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a746ec-5193-4d59-bc3c-5191a79f3bfc None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:06.442780 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a746ec-5193-4d59-bc3c-5191a79f3bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:06.443665 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a746ec-5193-4d59-bc3c-5191a79f3bfc None admin] PATH_INFO: `/v3/groups/e96cc682b6194c3480dd474da85b8bac` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:06.446887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a746ec-5193-4d59-bc3c-5191a79f3bfc None admin] RBAC: Authorizing `identity:get_group(group_id=e96cc682b6194c3480dd474da85b8bac)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:06.450115 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a746ec-5193-4d59-bc3c-5191a79f3bfc None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:06.454424 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 203/402] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 24 08:42:06 2022] GET /identity/v3/groups/e96cc682b6194c3480dd474da85b8bac => generated 226 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:06.462206 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5fe77d28-ee91-4034-bc28-03083f443dfe None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:06.462206 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5fe77d28-ee91-4034-bc28-03083f443dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:06.464381 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a746ec-5193-4d59-bc3c-5191a79f3bfc None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:06.514351 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:06.592534 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 200/403] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:06.592534 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe77d28-ee91-4034-bc28-03083f443dfe None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:06.683702 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a746ec-5193-4d59-bc3c-5191a79f3bfc None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:06.710604 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:06.743149 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd555251-454d-43a0-835c-39c666923a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:06.744121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd555251-454d-43a0-835c-39c666923a7f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:06.744553 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd555251-454d-43a0-835c-39c666923a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:06.745590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd555251-454d-43a0-835c-39c666923a7f None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:06.747031 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd555251-454d-43a0-835c-39c666923a7f None admin] RBAC: Authorizing `identity:get_project(project_id=9381518952174868a5bc99813df0342e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:06.749107 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd555251-454d-43a0-835c-39c666923a7f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:06.751887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 204/404] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:42:06 2022] GET /identity/v3/projects/9381518952174868a5bc99813df0342e => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:06.756159 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe77d28-ee91-4034-bc28-03083f443dfe None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:06.759719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd555251-454d-43a0-835c-39c666923a7f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:06.814953 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d367e99-4532-4b06-8f0a-945863b41f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:06.815926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d367e99-4532-4b06-8f0a-945863b41f19 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:06.816457 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d367e99-4532-4b06-8f0a-945863b41f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:06.816857 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d367e99-4532-4b06-8f0a-945863b41f19 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:06.836588 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d367e99-4532-4b06-8f0a-945863b41f19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:06.848407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-3d367e99-4532-4b06-8f0a-945863b41f19 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:42:06.848407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:06.848407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:06.848407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:06.848407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:06.848407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:06.848407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:06.848407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:06.848407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:06.848407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:06.848407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:06.848407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:06.848407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:06.848407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:06.848407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:06.848407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:06.849360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:06.849360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:06.849360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:06.849360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:06.849360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:06.849360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:06.849360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:06.849360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:06.849360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:06.849360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:06.849360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:06.849360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:06.849360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:06.849360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:06.849360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:06.849360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:06.850157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:06.850157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:06.850157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:06.850157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:06.850157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:06.850157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:06.850157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:06.850157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:06.850157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:06.850157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:06.850157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:42:06.850157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:06.852615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 201/405] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:42:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:06.857669 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d367e99-4532-4b06-8f0a-945863b41f19 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:06.927975 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:06.962275 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd555251-454d-43a0-835c-39c666923a7f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:07.041992 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068a062-640e-471b-9606-b677db31426c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:07.043266 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068a062-640e-471b-9606-b677db31426c None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:07.043970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068a062-640e-471b-9606-b677db31426c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:07.044300 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068a062-640e-471b-9606-b677db31426c None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e/groups/e96cc682b6194c3480dd474da85b8bac/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:07.046559 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068a062-640e-471b-9606-b677db31426c None admin] RBAC: Authorizing `identity:create_grant(project_id=9381518952174868a5bc99813df0342e, group_id=e96cc682b6194c3480dd474da85b8bac, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:07.048127 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068a062-640e-471b-9606-b677db31426c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:07.070234 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 205/406] 10.222.0.67 () {58 vars in 1442 bytes} [Wed Aug 24 08:42:06 2022] PUT /identity/v3/projects/9381518952174868a5bc99813df0342e/groups/e96cc682b6194c3480dd474da85b8bac/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:42:07.072213 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068a062-640e-471b-9606-b677db31426c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:07.086718 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d367e99-4532-4b06-8f0a-945863b41f19 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:07.135547 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea839a82-74e4-416f-805e-e48cf8adbba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:07.136411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea839a82-74e4-416f-805e-e48cf8adbba4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:07.136745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea839a82-74e4-416f-805e-e48cf8adbba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:07.137102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea839a82-74e4-416f-805e-e48cf8adbba4 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:07.137903 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea839a82-74e4-416f-805e-e48cf8adbba4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:07.138384 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea839a82-74e4-416f-805e-e48cf8adbba4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:07.140325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea839a82-74e4-416f-805e-e48cf8adbba4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:07.152459 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:07.155420 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 202/407] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:07.156916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409ab15-9d1f-4776-a1cd-4a5b3f73ca6c None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:07.157335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409ab15-9d1f-4776-a1cd-4a5b3f73ca6c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:07.157743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409ab15-9d1f-4776-a1cd-4a5b3f73ca6c None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:07.159732 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 203/408] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:07.165801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409ab15-9d1f-4776-a1cd-4a5b3f73ca6c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:07.242748 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068a062-640e-471b-9606-b677db31426c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:07.284994 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:07.299437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc8326-9ef4-4bdf-943b-8f3c4c0eb0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:07.300518 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc8326-9ef4-4bdf-943b-8f3c4c0eb0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:07.301117 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc8326-9ef4-4bdf-943b-8f3c4c0eb0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:07.301396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc8326-9ef4-4bdf-943b-8f3c4c0eb0c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:07.314349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409ab15-9d1f-4776-a1cd-4a5b3f73ca6c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:07.321237 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc8326-9ef4-4bdf-943b-8f3c4c0eb0c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:07.328195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-e4bc8326-9ef4-4bdf-943b-8f3c4c0eb0c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:42:07.328195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:07.328195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:07.328195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:07.328195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:07.328195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:07.328195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:07.328195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:07.328195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:07.328195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:07.328195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:07.328195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:07.328195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:07.328195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:07.328195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:07.328195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:07.329150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:07.329150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:07.329150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:07.329150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:07.329150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:07.329150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:07.329150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:07.329150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:07.329150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:07.329150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:07.329150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:07.329150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:07.329150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:07.329150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:07.329150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:07.329150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:07.329950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:07.329950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:07.329950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:07.329950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:07.329950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:07.329950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:07.329950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:07.329950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:07.329950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:07.329950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:07.329950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:42:07.329950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:07.329950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 206/409] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:42:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:07.335435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0dbc5-4e64-4122-b17f-9f31e2145099 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:07.335435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0dbc5-4e64-4122-b17f-9f31e2145099 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:07.335435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0dbc5-4e64-4122-b17f-9f31e2145099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:07.356238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f7e0dbc5-4e64-4122-b17f-9f31e2145099 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:07.357545 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f7e0dbc5-4e64-4122-b17f-9f31e2145099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:07.373702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5a17a-3a2c-474f-8e90-0932df9f54ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:07.375013 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5a17a-3a2c-474f-8e90-0932df9f54ce None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:07.375580 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5a17a-3a2c-474f-8e90-0932df9f54ce None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:07.376188 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5a17a-3a2c-474f-8e90-0932df9f54ce None admin] PATH_INFO: `/v3/groups/e96cc682b6194c3480dd474da85b8bac` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:07.378439 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5a17a-3a2c-474f-8e90-0932df9f54ce None admin] RBAC: Authorizing `identity:get_group(group_id=e96cc682b6194c3480dd474da85b8bac)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:07.381288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5a17a-3a2c-474f-8e90-0932df9f54ce None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:07.392024 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 204/410] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 24 08:42:07 2022] GET /identity/v3/groups/e96cc682b6194c3480dd474da85b8bac => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:07.393623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5a17a-3a2c-474f-8e90-0932df9f54ce None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:07.456498 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 207/411] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:07.458292 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0dbc5-4e64-4122-b17f-9f31e2145099 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:07.533319 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5a17a-3a2c-474f-8e90-0932df9f54ce None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:07.574623 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:07.578634 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1c7b9-a793-4bfd-9627-86a6f8cbffde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:07.579471 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1c7b9-a793-4bfd-9627-86a6f8cbffde None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:07.579757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1c7b9-a793-4bfd-9627-86a6f8cbffde None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:07.580029 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1c7b9-a793-4bfd-9627-86a6f8cbffde None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:07.580739 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1c7b9-a793-4bfd-9627-86a6f8cbffde None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:07.581045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1c7b9-a793-4bfd-9627-86a6f8cbffde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:07.582421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1c7b9-a793-4bfd-9627-86a6f8cbffde None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:07.593540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 205/412] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:42:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:07.599208 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1c7b9-a793-4bfd-9627-86a6f8cbffde None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:07.614736 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:07.726193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1c7b9-a793-4bfd-9627-86a6f8cbffde None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:07.766078 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b76b5-7153-49e4-ac19-3ddeca586d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:07.766739 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b76b5-7153-49e4-ac19-3ddeca586d30 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:07.767021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b76b5-7153-49e4-ac19-3ddeca586d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:07.767294 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b76b5-7153-49e4-ac19-3ddeca586d30 None admin] PATH_INFO: `/v3/groups/612063f7d74f41b79d34ab45962a78fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:07.768814 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b76b5-7153-49e4-ac19-3ddeca586d30 None admin] RBAC: Authorizing `identity:get_group(group_id=612063f7d74f41b79d34ab45962a78fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:07.770836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b76b5-7153-49e4-ac19-3ddeca586d30 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:07.773417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 206/413] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 24 08:42:07 2022] GET /identity/v3/groups/612063f7d74f41b79d34ab45962a78fd => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:07.779055 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b76b5-7153-49e4-ac19-3ddeca586d30 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:07.806117 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0dbc5-4e64-4122-b17f-9f31e2145099 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:07.875533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:07.902954 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b76b5-7153-49e4-ac19-3ddeca586d30 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:07.912829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce320190-0862-4115-bc03-3a9f8da6be55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:07.914490 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce320190-0862-4115-bc03-3a9f8da6be55 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:07.914490 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce320190-0862-4115-bc03-3a9f8da6be55 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:07.914922 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce320190-0862-4115-bc03-3a9f8da6be55 None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:07.920899 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce320190-0862-4115-bc03-3a9f8da6be55 None admin] RBAC: Authorizing `identity:get_project(project_id=9381518952174868a5bc99813df0342e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:07.922503 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce320190-0862-4115-bc03-3a9f8da6be55 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:07.925310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 208/414] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:42:07 2022] GET /identity/v3/projects/9381518952174868a5bc99813df0342e => generated 302 bytes in 468 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:07.931078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce320190-0862-4115-bc03-3a9f8da6be55 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:07.943677 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f120c1-6b50-4e32-b24c-d22253e78c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:07.945215 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f120c1-6b50-4e32-b24c-d22253e78c40 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:07.945643 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f120c1-6b50-4e32-b24c-d22253e78c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:07.946567 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f120c1-6b50-4e32-b24c-d22253e78c40 None admin] PATH_INFO: `/v3/projects/456ae4386c984b51b020a86192cad430` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:07.947910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f120c1-6b50-4e32-b24c-d22253e78c40 None admin] RBAC: Authorizing `identity:get_project(project_id=456ae4386c984b51b020a86192cad430)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:07.949538 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f120c1-6b50-4e32-b24c-d22253e78c40 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:07.956488 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 207/415] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:42:07 2022] GET /identity/v3/projects/456ae4386c984b51b020a86192cad430 => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:07.958380 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f120c1-6b50-4e32-b24c-d22253e78c40 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:08.063113 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce320190-0862-4115-bc03-3a9f8da6be55 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:08.093267 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f120c1-6b50-4e32-b24c-d22253e78c40 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:08.101063 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:08.102487 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790f778-e42c-4c82-89db-21d3b6074138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:08.103207 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790f778-e42c-4c82-89db-21d3b6074138 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:08.103488 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790f778-e42c-4c82-89db-21d3b6074138 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:08.103806 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790f778-e42c-4c82-89db-21d3b6074138 None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e/groups/e96cc682b6194c3480dd474da85b8bac/roles/5915b1954b514151ab17ff5d531036e4` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:08.106786 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790f778-e42c-4c82-89db-21d3b6074138 None admin] RBAC: Authorizing `identity:create_grant(project_id=9381518952174868a5bc99813df0342e, group_id=e96cc682b6194c3480dd474da85b8bac, role_id=5915b1954b514151ab17ff5d531036e4)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:08.108315 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790f778-e42c-4c82-89db-21d3b6074138 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:08.128186 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 209/416] 10.222.0.67 () {58 vars in 1442 bytes} [Wed Aug 24 08:42:07 2022] PUT /identity/v3/projects/9381518952174868a5bc99813df0342e/groups/e96cc682b6194c3480dd474da85b8bac/roles/5915b1954b514151ab17ff5d531036e4 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:42:08.129639 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790f778-e42c-4c82-89db-21d3b6074138 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:08.163988 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b2788-2195-4ba8-b57e-f5765af156d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:08.166424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b2788-2195-4ba8-b57e-f5765af156d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:08.167832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b2788-2195-4ba8-b57e-f5765af156d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:08.168407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b2788-2195-4ba8-b57e-f5765af156d7 None admin] PATH_INFO: `/v3/projects/456ae4386c984b51b020a86192cad430/groups/612063f7d74f41b79d34ab45962a78fd/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:08.171923 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b2788-2195-4ba8-b57e-f5765af156d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=456ae4386c984b51b020a86192cad430, group_id=612063f7d74f41b79d34ab45962a78fd, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:08.176378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b2788-2195-4ba8-b57e-f5765af156d7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:08.203014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 208/417] 10.222.0.67 () {58 vars in 1442 bytes} [Wed Aug 24 08:42:07 2022] PUT /identity/v3/projects/456ae4386c984b51b020a86192cad430/groups/612063f7d74f41b79d34ab45962a78fd/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:42:08.339164 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790f778-e42c-4c82-89db-21d3b6074138 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:08.386799 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd63fe-5d0b-48f0-a7a7-51803f444062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:08.387840 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd63fe-5d0b-48f0-a7a7-51803f444062 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:08.388367 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd63fe-5d0b-48f0-a7a7-51803f444062 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:08.388896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd63fe-5d0b-48f0-a7a7-51803f444062 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:08.402717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd63fe-5d0b-48f0-a7a7-51803f444062 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:08.409139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-2fbd63fe-5d0b-48f0-a7a7-51803f444062 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:42:08.409139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:08.409139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:08.409139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:08.409139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:08.409139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:08.409139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:08.409139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:08.409139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:08.409139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:08.409139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:08.409139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:08.409139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:08.409139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:08.409139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:08.409139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:08.410202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:08.410202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:08.410202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:08.410202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:08.410202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:08.410202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:08.410202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:08.410202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:08.410202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:08.410202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:08.410202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:08.410202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:08.410202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:08.410202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:08.410202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:08.410202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:08.410996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:08.410996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:08.410996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:08.410996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:08.410996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:08.410996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:08.410996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:08.410996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:08.410996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:08.410996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:08.410996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:42:08.410996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:08.413868 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 210/418] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:42:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:08.419798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b2788-2195-4ba8-b57e-f5765af156d7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:08.431607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:08.572760 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b2788-2195-4ba8-b57e-f5765af156d7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:08.631054 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a9d84-9ec9-443d-9bf5-b85332d38a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:08.632650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a9d84-9ec9-443d-9bf5-b85332d38a23 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:08.633139 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a9d84-9ec9-443d-9bf5-b85332d38a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:08.633531 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a9d84-9ec9-443d-9bf5-b85332d38a23 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:08.634665 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a9d84-9ec9-443d-9bf5-b85332d38a23 None admin] RBAC: Adding query filter params (name=member) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:08.635162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a9d84-9ec9-443d-9bf5-b85332d38a23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:08.637696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a9d84-9ec9-443d-9bf5-b85332d38a23 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:08.655662 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 209/419] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:42:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:08.661897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd63fe-5d0b-48f0-a7a7-51803f444062 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:08.676465 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eecc11-27a6-4aaf-8822-2b90142fa181 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:08.676922 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eecc11-27a6-4aaf-8822-2b90142fa181 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:08.677372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eecc11-27a6-4aaf-8822-2b90142fa181 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:08.680330 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 210/420] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:08.689960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2222552-f92b-4b52-a34b-82e4da936e6a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:08.690268 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2222552-f92b-4b52-a34b-82e4da936e6a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:08.690524 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2222552-f92b-4b52-a34b-82e4da936e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:08.705457 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:08.725330 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a2222552-f92b-4b52-a34b-82e4da936e6a None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:08.727234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a2222552-f92b-4b52-a34b-82e4da936e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:08.795863 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd63fe-5d0b-48f0-a7a7-51803f444062 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:08.836273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d293e6-9388-45c7-a082-4b9b21195a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:08.836917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d293e6-9388-45c7-a082-4b9b21195a10 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:08.837204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d293e6-9388-45c7-a082-4b9b21195a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:08.837617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d293e6-9388-45c7-a082-4b9b21195a10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:08.889607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:08.900401 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 211/421] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:08.940039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d293e6-9388-45c7-a082-4b9b21195a10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:08.941550 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d293e6-9388-45c7-a082-4b9b21195a10 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:08.989251 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:09.090128 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 211/422] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:42:09.098214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2222552-f92b-4b52-a34b-82e4da936e6a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:09.237291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2222552-f92b-4b52-a34b-82e4da936e6a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:09.279869 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cff725-6a05-4a88-8d30-f1fe63fd9277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:09.280574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cff725-6a05-4a88-8d30-f1fe63fd9277 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:09.280851 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cff725-6a05-4a88-8d30-f1fe63fd9277 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:09.281038 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d293e6-9388-45c7-a082-4b9b21195a10 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:09.281141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cff725-6a05-4a88-8d30-f1fe63fd9277 None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:09.282259 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cff725-6a05-4a88-8d30-f1fe63fd9277 None admin] RBAC: Authorizing `identity:get_project(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:09.283840 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cff725-6a05-4a88-8d30-f1fe63fd9277 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:09.285888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 212/423] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:42:09 2022] GET /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:09.292708 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cff725-6a05-4a88-8d30-f1fe63fd9277 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:09.367598 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:09.411025 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cff725-6a05-4a88-8d30-f1fe63fd9277 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:09.455939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45943f45-f039-4ac5-af5a-dfe48ba9d73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:09.456964 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45943f45-f039-4ac5-af5a-dfe48ba9d73e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:09.457375 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45943f45-f039-4ac5-af5a-dfe48ba9d73e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:09.457641 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45943f45-f039-4ac5-af5a-dfe48ba9d73e None admin] PATH_INFO: `/v3/groups/e96cc682b6194c3480dd474da85b8bac` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:09.458805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45943f45-f039-4ac5-af5a-dfe48ba9d73e None admin] RBAC: Authorizing `identity:get_group(group_id=e96cc682b6194c3480dd474da85b8bac)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:09.461075 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45943f45-f039-4ac5-af5a-dfe48ba9d73e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:09.465035 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 213/424] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 24 08:42:09 2022] GET /identity/v3/groups/e96cc682b6194c3480dd474da85b8bac => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:09.468322 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986423cf-95e6-4908-8956-8a7e04fe8964 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:09.468665 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986423cf-95e6-4908-8956-8a7e04fe8964 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:09.468940 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986423cf-95e6-4908-8956-8a7e04fe8964 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:09.470678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 214/425] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:09.472542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986423cf-95e6-4908-8956-8a7e04fe8964 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:09.495465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d293e6-9388-45c7-a082-4b9b21195a10 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:09.522190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:09.535601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb93039-b313-451f-b705-d0cc115440df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:09.536529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb93039-b313-451f-b705-d0cc115440df None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:09.536958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb93039-b313-451f-b705-d0cc115440df None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:09.537756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb93039-b313-451f-b705-d0cc115440df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:09.550945 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb93039-b313-451f-b705-d0cc115440df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:09.556302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-adb93039-b313-451f-b705-d0cc115440df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:42:09.556302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:09.556302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:09.556302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:09.556302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:09.556302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:09.556302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:09.556302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:09.556302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:09.556302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:09.556302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:09.556302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:09.556302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:09.556302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:09.556302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:09.556302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:09.557397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:09.557397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:09.557397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:09.557397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:09.557397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:09.557397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:09.557397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:09.557397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:09.557397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:09.557397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:09.557397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:09.557397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:09.557397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:09.557397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:09.557397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:09.557397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:09.558310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:09.558310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:09.558310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:09.558310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:09.558310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:09.558310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:09.558310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:09.558310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:09.558310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:09.558310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:09.558310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:42:09.558310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:09.560640 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 212/426] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:42:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:09.562105 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5487c6d-c0eb-4667-9af7-fcceb91b40f9 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:09.562603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5487c6d-c0eb-4667-9af7-fcceb91b40f9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:09.563082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5487c6d-c0eb-4667-9af7-fcceb91b40f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:09.583838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b5487c6d-c0eb-4667-9af7-fcceb91b40f9 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:09.585804 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b5487c6d-c0eb-4667-9af7-fcceb91b40f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:09.589480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986423cf-95e6-4908-8956-8a7e04fe8964 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:09.637873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826dda61-0be4-45ce-9631-f4ba79c52e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:09.639047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dda61-0be4-45ce-9631-f4ba79c52e84 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:09.639483 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dda61-0be4-45ce-9631-f4ba79c52e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:09.639945 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dda61-0be4-45ce-9631-f4ba79c52e84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:09.641311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826dda61-0be4-45ce-9631-f4ba79c52e84 None admin] RBAC: Adding query filter params (group.id=e96cc682b6194c3480dd474da85b8bac, role.id=92533f29c1d54fe09591e6f2a906ac04, scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:09.641836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826dda61-0be4-45ce-9631-f4ba79c52e84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:09.644847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826dda61-0be4-45ce-9631-f4ba79c52e84 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:09.649789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:09.664626 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 215/427] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 24 08:42:09 2022] GET /identity/v3/role_assignments?group.id=e96cc682b6194c3480dd474da85b8bac&scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d&role.id=92533f29c1d54fe09591e6f2a906ac04 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:09.666657 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826dda61-0be4-45ce-9631-f4ba79c52e84 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:09.671475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 213/428] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:09.868642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826dda61-0be4-45ce-9631-f4ba79c52e84 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:09.906942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f319cde-f4a0-4f1f-9f4f-5cf0a6b1a3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:09.907580 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f319cde-f4a0-4f1f-9f4f-5cf0a6b1a3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:09.907826 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f319cde-f4a0-4f1f-9f4f-5cf0a6b1a3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:09.908118 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f319cde-f4a0-4f1f-9f4f-5cf0a6b1a3a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:09.908822 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f319cde-f4a0-4f1f-9f4f-5cf0a6b1a3a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:09.909121 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f319cde-f4a0-4f1f-9f4f-5cf0a6b1a3a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:09.910496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f319cde-f4a0-4f1f-9f4f-5cf0a6b1a3a2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:09.919625 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 216/429] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:09.926009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5487c6d-c0eb-4667-9af7-fcceb91b40f9 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:10.056247 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5487c6d-c0eb-4667-9af7-fcceb91b40f9 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:10.060716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f319cde-f4a0-4f1f-9f4f-5cf0a6b1a3a2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:10.073250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:10.097905 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d6eb7-483b-4a9f-8357-cea5a861170a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:10.098577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d6eb7-483b-4a9f-8357-cea5a861170a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:10.098816 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d6eb7-483b-4a9f-8357-cea5a861170a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:10.099076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d6eb7-483b-4a9f-8357-cea5a861170a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:10.135527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:10.186986 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f319cde-f4a0-4f1f-9f4f-5cf0a6b1a3a2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:10.189239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d6eb7-483b-4a9f-8357-cea5a861170a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:10.190915 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d6eb7-483b-4a9f-8357-cea5a861170a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:10.227735 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b970cf-5c47-4158-9b07-96cdf81ae0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:10.228593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b970cf-5c47-4158-9b07-96cdf81ae0a2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:10.228972 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b970cf-5c47-4158-9b07-96cdf81ae0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:10.229340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b970cf-5c47-4158-9b07-96cdf81ae0a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:10.245026 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b970cf-5c47-4158-9b07-96cdf81ae0a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:10.250583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-c4b970cf-5c47-4158-9b07-96cdf81ae0a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:42:10.250583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:10.250583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:10.250583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:10.250583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:10.250583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:10.250583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:10.250583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:10.250583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:10.250583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:10.250583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:10.250583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:10.250583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:10.250583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:10.250583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:10.250583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:10.251551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:10.251551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:10.251551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:10.251551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:10.251551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:10.251551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:10.251551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:10.251551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:10.251551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:10.251551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:10.251551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:10.251551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:10.251551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:10.251551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:10.251551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:10.251551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:10.252684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:10.252684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:10.252684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:10.252684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:10.252684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:10.252684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:10.252684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:10.252684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:10.252684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:10.252684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:10.252684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 08:42:10.252684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:10.252684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 217/430] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:42:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:42:10.259088 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b970cf-5c47-4158-9b07-96cdf81ae0a2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:10.325789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 214/431] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:42:10.333309 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d6eb7-483b-4a9f-8357-cea5a861170a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:10.352824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:10.446320 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b970cf-5c47-4158-9b07-96cdf81ae0a2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:10.479784 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:10.502166 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d6eb7-483b-4a9f-8357-cea5a861170a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:10.503322 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143b7b3-81fe-4a05-99ec-802400357404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:10.504501 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143b7b3-81fe-4a05-99ec-802400357404 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:10.504984 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143b7b3-81fe-4a05-99ec-802400357404 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:10.505449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143b7b3-81fe-4a05-99ec-802400357404 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:10.508131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143b7b3-81fe-4a05-99ec-802400357404 None admin] RBAC: Adding query filter params (name=member) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:10.508842 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143b7b3-81fe-4a05-99ec-802400357404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:10.510615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143b7b3-81fe-4a05-99ec-802400357404 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:10.527276 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 218/432] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:42:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:10.533363 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143b7b3-81fe-4a05-99ec-802400357404 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:10.558042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36893122-9e97-4f78-ab18-cec5b019d07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:10.558690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36893122-9e97-4f78-ab18-cec5b019d07f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:10.558971 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36893122-9e97-4f78-ab18-cec5b019d07f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:10.559243 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36893122-9e97-4f78-ab18-cec5b019d07f None admin] PATH_INFO: `/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:10.561057 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36893122-9e97-4f78-ab18-cec5b019d07f None admin] RBAC: Authorizing `identity:get_project(project_id=ef0fcc9b994c4325b9eeb9bdd6070d1d)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:10.563151 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36893122-9e97-4f78-ab18-cec5b019d07f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:10.566098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 215/433] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:42:10 2022] GET /identity/v3/projects/ef0fcc9b994c4325b9eeb9bdd6070d1d => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:10.573186 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36893122-9e97-4f78-ab18-cec5b019d07f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:10.671077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143b7b3-81fe-4a05-99ec-802400357404 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:10.695303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:10.721917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c92f5-1231-436c-a207-54e0958694f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:10.722790 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c92f5-1231-436c-a207-54e0958694f0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:10.723221 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c92f5-1231-436c-a207-54e0958694f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:10.723686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c92f5-1231-436c-a207-54e0958694f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:10.725132 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36893122-9e97-4f78-ab18-cec5b019d07f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:10.783109 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869aeb2-7a33-40b6-9e48-f74056147546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:10.784300 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869aeb2-7a33-40b6-9e48-f74056147546 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:10.784809 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869aeb2-7a33-40b6-9e48-f74056147546 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:10.785145 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869aeb2-7a33-40b6-9e48-f74056147546 None admin] PATH_INFO: `/v3/groups/e96cc682b6194c3480dd474da85b8bac` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:10.786445 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869aeb2-7a33-40b6-9e48-f74056147546 None admin] RBAC: Authorizing `identity:get_group(group_id=e96cc682b6194c3480dd474da85b8bac)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:10.788804 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869aeb2-7a33-40b6-9e48-f74056147546 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:10.791066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 216/434] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 24 08:42:10 2022] GET /identity/v3/groups/e96cc682b6194c3480dd474da85b8bac => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:10.792175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1fc78-24b4-4394-a5e1-3eb58e98cd31 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:10.792451 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1fc78-24b4-4394-a5e1-3eb58e98cd31 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:10.792720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1fc78-24b4-4394-a5e1-3eb58e98cd31 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:10.794247 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 217/435] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:10.800263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d762bc-5442-4698-8440-bc4d945fd0ae None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:10.800627 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d762bc-5442-4698-8440-bc4d945fd0ae None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:10.801008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d762bc-5442-4698-8440-bc4d945fd0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:10.803679 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:10.829497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-05d762bc-5442-4698-8440-bc4d945fd0ae None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:10.830045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c92f5-1231-436c-a207-54e0958694f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:10.830752 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-05d762bc-5442-4698-8440-bc4d945fd0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:10.831528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c92f5-1231-436c-a207-54e0958694f0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:10.924408 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 218/436] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:42:10.924408 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d762bc-5442-4698-8440-bc4d945fd0ae None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:10.971249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 219/437] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:42:10.971249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ca838-2b5e-4c03-b2b5-073540cf7cac None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:10.971249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ca838-2b5e-4c03-b2b5-073540cf7cac None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:10.971249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ca838-2b5e-4c03-b2b5-073540cf7cac None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:10.971249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 220/438] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:10.975007 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ca838-2b5e-4c03-b2b5-073540cf7cac None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:11.022351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:11.056808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d762bc-5442-4698-8440-bc4d945fd0ae None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:11.096541 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:11.106608 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537f2ec-1660-4174-bd4e-05ec9aca599d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:11.108264 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537f2ec-1660-4174-bd4e-05ec9aca599d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:11.108728 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537f2ec-1660-4174-bd4e-05ec9aca599d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:11.109361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537f2ec-1660-4174-bd4e-05ec9aca599d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:11.110812 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537f2ec-1660-4174-bd4e-05ec9aca599d None admin] RBAC: Adding query filter params (group.id=e96cc682b6194c3480dd474da85b8bac, role.id=5915b1954b514151ab17ff5d531036e4, scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:11.111587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537f2ec-1660-4174-bd4e-05ec9aca599d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:11.113865 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537f2ec-1660-4174-bd4e-05ec9aca599d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:11.122011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ca838-2b5e-4c03-b2b5-073540cf7cac None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:11.122962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 219/439] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 24 08:42:10 2022] GET /identity/v3/role_assignments?group.id=e96cc682b6194c3480dd474da85b8bac&scope.project.id=ef0fcc9b994c4325b9eeb9bdd6070d1d&role.id=5915b1954b514151ab17ff5d531036e4 => generated 611 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:11.125641 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccc6af-f225-4083-b8d7-ef32deda6d86 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:11.126196 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccc6af-f225-4083-b8d7-ef32deda6d86 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:11.126607 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccc6af-f225-4083-b8d7-ef32deda6d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:11.160924 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-77ccc6af-f225-4083-b8d7-ef32deda6d86 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:11.162328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-77ccc6af-f225-4083-b8d7-ef32deda6d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:11.170610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca15afc-db0e-4fec-9e49-7b4272730ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:11.171531 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca15afc-db0e-4fec-9e49-7b4272730ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:11.171992 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca15afc-db0e-4fec-9e49-7b4272730ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:11.172526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca15afc-db0e-4fec-9e49-7b4272730ac8 None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:11.173815 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca15afc-db0e-4fec-9e49-7b4272730ac8 None admin] RBAC: Authorizing `identity:get_project(project_id=9381518952174868a5bc99813df0342e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:11.175588 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca15afc-db0e-4fec-9e49-7b4272730ac8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:11.177862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 221/440] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:42:10 2022] GET /identity/v3/projects/9381518952174868a5bc99813df0342e => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:11.184310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca15afc-db0e-4fec-9e49-7b4272730ac8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:11.251068 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 220/441] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:11.252020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ccc6af-f225-4083-b8d7-ef32deda6d86 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:11.324842 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca15afc-db0e-4fec-9e49-7b4272730ac8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:11.363106 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:11.374745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a0763-8eff-4fa0-a676-e4b7e8d69c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:11.376005 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a0763-8eff-4fa0-a676-e4b7e8d69c27 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:11.376366 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a0763-8eff-4fa0-a676-e4b7e8d69c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:11.376646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a0763-8eff-4fa0-a676-e4b7e8d69c27 None admin] PATH_INFO: `/v3/groups/e96cc682b6194c3480dd474da85b8bac` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:11.377835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a0763-8eff-4fa0-a676-e4b7e8d69c27 None admin] RBAC: Authorizing `identity:get_group(group_id=e96cc682b6194c3480dd474da85b8bac)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:11.379301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a0763-8eff-4fa0-a676-e4b7e8d69c27 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:11.381161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 222/442] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 24 08:42:11 2022] GET /identity/v3/groups/e96cc682b6194c3480dd474da85b8bac => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:11.387135 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a0763-8eff-4fa0-a676-e4b7e8d69c27 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:11.394632 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ccc6af-f225-4083-b8d7-ef32deda6d86 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:11.415611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:11.443774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664b97c-fe97-4285-9cca-ba296d10c35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:11.444651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664b97c-fe97-4285-9cca-ba296d10c35d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:11.444936 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664b97c-fe97-4285-9cca-ba296d10c35d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:11.445209 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664b97c-fe97-4285-9cca-ba296d10c35d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:11.465023 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664b97c-fe97-4285-9cca-ba296d10c35d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:11.470446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-e664b97c-fe97-4285-9cca-ba296d10c35d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:42:11.470446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:11.470446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:11.470446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:11.470446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:11.470446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:11.470446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:11.470446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:11.470446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:11.470446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:11.470446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:11.470446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:11.470446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:11.470446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:11.470446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:11.470446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:11.471399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:11.471399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:11.471399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:11.471399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:11.471399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:11.471399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:11.471399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:11.471399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:11.471399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:11.471399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:11.471399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:11.471399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:11.471399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:11.471399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:11.471399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:11.471399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:11.472412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:11.472412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:11.472412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:11.472412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:11.472412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:11.472412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:11.472412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:11.472412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:11.472412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:11.472412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:11.472412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:42:11.472412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:11.472412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 221/443] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:42:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:11.479998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664b97c-fe97-4285-9cca-ba296d10c35d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:11.529200 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a0763-8eff-4fa0-a676-e4b7e8d69c27 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:11.566547 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4b66f-0139-4875-a12e-63b0319fedbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:11.567165 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4b66f-0139-4875-a12e-63b0319fedbd None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:11.567436 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4b66f-0139-4875-a12e-63b0319fedbd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:11.567701 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4b66f-0139-4875-a12e-63b0319fedbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:11.568463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4b66f-0139-4875-a12e-63b0319fedbd None admin] RBAC: Adding query filter params (group.id=e96cc682b6194c3480dd474da85b8bac, role.id=92533f29c1d54fe09591e6f2a906ac04, scope.project.id=9381518952174868a5bc99813df0342e) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:11.568768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4b66f-0139-4875-a12e-63b0319fedbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:11.570767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4b66f-0139-4875-a12e-63b0319fedbd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:11.577882 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 223/444] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 24 08:42:11 2022] GET /identity/v3/role_assignments?group.id=e96cc682b6194c3480dd474da85b8bac&scope.project.id=9381518952174868a5bc99813df0342e&role.id=92533f29c1d54fe09591e6f2a906ac04 => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:11.578693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4b66f-0139-4875-a12e-63b0319fedbd None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:11.645400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664b97c-fe97-4285-9cca-ba296d10c35d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:11.687992 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b31483-d969-4be4-be41-bdda03f001c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:11.688724 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b31483-d969-4be4-be41-bdda03f001c7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:11.689003 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b31483-d969-4be4-be41-bdda03f001c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:11.689272 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b31483-d969-4be4-be41-bdda03f001c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:11.689966 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b31483-d969-4be4-be41-bdda03f001c7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:11.690800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b31483-d969-4be4-be41-bdda03f001c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:11.692272 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b31483-d969-4be4-be41-bdda03f001c7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:11.693938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:11.702707 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:11.703387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 222/445] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:11.709145 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b31483-d969-4be4-be41-bdda03f001c7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:11.734592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4b66f-0139-4875-a12e-63b0319fedbd None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:11.785040 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe416d57-0731-4e37-bb86-00056ccde7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:11.785613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe416d57-0731-4e37-bb86-00056ccde7df None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:11.785894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe416d57-0731-4e37-bb86-00056ccde7df None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:11.786170 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe416d57-0731-4e37-bb86-00056ccde7df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:11.800826 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe416d57-0731-4e37-bb86-00056ccde7df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:11.809526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-fe416d57-0731-4e37-bb86-00056ccde7df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:42:11.809526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:11.809526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:11.809526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:11.809526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:11.809526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:11.809526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:11.809526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:11.809526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:11.809526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:11.809526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:11.809526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:11.809526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:11.809526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:11.809526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:11.809526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:11.810482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:11.810482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:11.810482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:11.810482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:11.810482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:11.810482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:11.810482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:11.810482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:11.810482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:11.810482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:11.810482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:11.810482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:11.810482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:11.810482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:11.810482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:11.810482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:11.811282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:11.811282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:11.811282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:11.811282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:11.811282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:11.811282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:11.811282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:11.811282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:11.811282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:11.811282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:11.811282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:42:11.811282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:11.811282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 224/446] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:42:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:11.816705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe416d57-0731-4e37-bb86-00056ccde7df None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:11.856220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b31483-d969-4be4-be41-bdda03f001c7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:11.909107 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e771550-d531-4c63-81a5-24d3816dc434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:11.910031 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e771550-d531-4c63-81a5-24d3816dc434 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:11.910485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e771550-d531-4c63-81a5-24d3816dc434 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:11.910934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e771550-d531-4c63-81a5-24d3816dc434 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:11.957692 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe416d57-0731-4e37-bb86-00056ccde7df None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:12.001634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:12.005826 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e7ad3-7628-48b1-a652-f50a239ae631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:12.007011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e7ad3-7628-48b1-a652-f50a239ae631 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:12.007514 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e7ad3-7628-48b1-a652-f50a239ae631 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:12.008010 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e7ad3-7628-48b1-a652-f50a239ae631 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:12.009275 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e771550-d531-4c63-81a5-24d3816dc434 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:12.009380 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e7ad3-7628-48b1-a652-f50a239ae631 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:12.009939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e7ad3-7628-48b1-a652-f50a239ae631 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:12.010752 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e771550-d531-4c63-81a5-24d3816dc434 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:12.012643 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e7ad3-7628-48b1-a652-f50a239ae631 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:12.027147 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 225/447] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:42:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:12.032742 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e7ad3-7628-48b1-a652-f50a239ae631 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:12.036466 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:12.117475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 223/448] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:42:12.123486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e771550-d531-4c63-81a5-24d3816dc434 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:12.144291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e7ad3-7628-48b1-a652-f50a239ae631 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:12.181176 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a25907-8ac1-4156-9416-0c7addd1f205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:12.181787 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a25907-8ac1-4156-9416-0c7addd1f205 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:12.182065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a25907-8ac1-4156-9416-0c7addd1f205 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:12.182337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a25907-8ac1-4156-9416-0c7addd1f205 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:12.226162 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:12.232640 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e771550-d531-4c63-81a5-24d3816dc434 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:12.251666 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a25907-8ac1-4156-9416-0c7addd1f205 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:12.253237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a25907-8ac1-4156-9416-0c7addd1f205 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:12.256370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:12.270661 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a4edb-b585-4c6f-be2c-3073b1d553c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:12.271307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a4edb-b585-4c6f-be2c-3073b1d553c0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:12.271586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a4edb-b585-4c6f-be2c-3073b1d553c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:12.271855 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a4edb-b585-4c6f-be2c-3073b1d553c0 None admin] PATH_INFO: `/v3/projects/9381518952174868a5bc99813df0342e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:12.273625 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a4edb-b585-4c6f-be2c-3073b1d553c0 None admin] RBAC: Authorizing `identity:get_project(project_id=9381518952174868a5bc99813df0342e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:12.275077 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a4edb-b585-4c6f-be2c-3073b1d553c0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:12.276760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 224/449] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:42:12 2022] GET /identity/v3/projects/9381518952174868a5bc99813df0342e => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:12.282371 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a4edb-b585-4c6f-be2c-3073b1d553c0 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:12.357635 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 226/450] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:42:12.363263 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a25907-8ac1-4156-9416-0c7addd1f205 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:12.384542 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a4edb-b585-4c6f-be2c-3073b1d553c0 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:12.418878 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c7055-a966-4e72-9844-f76e3c2e8c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:12.419499 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c7055-a966-4e72-9844-f76e3c2e8c56 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:12.419782 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c7055-a966-4e72-9844-f76e3c2e8c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:12.420048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c7055-a966-4e72-9844-f76e3c2e8c56 None admin] PATH_INFO: `/v3/groups/e96cc682b6194c3480dd474da85b8bac` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:12.421190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c7055-a966-4e72-9844-f76e3c2e8c56 None admin] RBAC: Authorizing `identity:get_group(group_id=e96cc682b6194c3480dd474da85b8bac)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:12.422637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c7055-a966-4e72-9844-f76e3c2e8c56 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:12.424239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 225/451] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 24 08:42:12 2022] GET /identity/v3/groups/e96cc682b6194c3480dd474da85b8bac => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:12.430633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c7055-a966-4e72-9844-f76e3c2e8c56 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:12.462039 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a25907-8ac1-4156-9416-0c7addd1f205 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:12.463535 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:12.464495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:12.497084 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc80f1-de74-495a-b3e6-1f24ea089c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:12.497715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc80f1-de74-495a-b3e6-1f24ea089c4b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:12.497993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc80f1-de74-495a-b3e6-1f24ea089c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:12.498266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc80f1-de74-495a-b3e6-1f24ea089c4b None admin] PATH_INFO: `/v3/projects/456ae4386c984b51b020a86192cad430` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:12.499646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc80f1-de74-495a-b3e6-1f24ea089c4b None admin] RBAC: Authorizing `identity:get_project(project_id=456ae4386c984b51b020a86192cad430)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:12.501127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc80f1-de74-495a-b3e6-1f24ea089c4b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:12.502750 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 227/452] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:42:12 2022] GET /identity/v3/projects/456ae4386c984b51b020a86192cad430 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:12.508917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc80f1-de74-495a-b3e6-1f24ea089c4b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:12.536747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c7055-a966-4e72-9844-f76e3c2e8c56 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:12.572185 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb18520-5604-4a0e-9d5b-d45578f5194b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:12.572788 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18520-5604-4a0e-9d5b-d45578f5194b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:12.573071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18520-5604-4a0e-9d5b-d45578f5194b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:12.573343 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18520-5604-4a0e-9d5b-d45578f5194b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:12.574070 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb18520-5604-4a0e-9d5b-d45578f5194b None admin] RBAC: Adding query filter params (group.id=e96cc682b6194c3480dd474da85b8bac, role.id=5915b1954b514151ab17ff5d531036e4, scope.project.id=9381518952174868a5bc99813df0342e) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:12.574380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb18520-5604-4a0e-9d5b-d45578f5194b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:12.575829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb18520-5604-4a0e-9d5b-d45578f5194b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:12.581785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 226/453] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 24 08:42:12 2022] GET /identity/v3/role_assignments?group.id=e96cc682b6194c3480dd474da85b8bac&scope.project.id=9381518952174868a5bc99813df0342e&role.id=5915b1954b514151ab17ff5d531036e4 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:12.615194 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc80f1-de74-495a-b3e6-1f24ea089c4b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:12.653441 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257f24d-ce2c-46e2-9eed-eeea62ce4beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:12.654089 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257f24d-ce2c-46e2-9eed-eeea62ce4beb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:12.654369 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257f24d-ce2c-46e2-9eed-eeea62ce4beb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:12.654641 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257f24d-ce2c-46e2-9eed-eeea62ce4beb None admin] PATH_INFO: `/v3/groups/612063f7d74f41b79d34ab45962a78fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:12.655823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257f24d-ce2c-46e2-9eed-eeea62ce4beb None admin] RBAC: Authorizing `identity:get_group(group_id=612063f7d74f41b79d34ab45962a78fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:12.657327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257f24d-ce2c-46e2-9eed-eeea62ce4beb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:12.658969 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 228/454] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 24 08:42:12 2022] GET /identity/v3/groups/612063f7d74f41b79d34ab45962a78fd => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:12.665419 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb18520-5604-4a0e-9d5b-d45578f5194b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:12.767523 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:12.773981 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb18520-5604-4a0e-9d5b-d45578f5194b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:12.809834 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b718d72-e34b-4aaa-b0c8-d02169c1d842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:12.810450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b718d72-e34b-4aaa-b0c8-d02169c1d842 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:12.810727 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b718d72-e34b-4aaa-b0c8-d02169c1d842 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:12.810996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b718d72-e34b-4aaa-b0c8-d02169c1d842 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:12.811734 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b718d72-e34b-4aaa-b0c8-d02169c1d842 None admin] RBAC: Adding query filter params (group.id=612063f7d74f41b79d34ab45962a78fd, role.id=b13ffa72344c4856901b1ba003e273fd, scope.project.id=456ae4386c984b51b020a86192cad430) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:12.812046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b718d72-e34b-4aaa-b0c8-d02169c1d842 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:12.813528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b718d72-e34b-4aaa-b0c8-d02169c1d842 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:12.820670 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 227/455] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 24 08:42:12 2022] GET /identity/v3/role_assignments?group.id=612063f7d74f41b79d34ab45962a78fd&scope.project.id=456ae4386c984b51b020a86192cad430&role.id=b13ffa72344c4856901b1ba003e273fd => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:15.221339 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1f145-6cfc-4ece-9101-c9f19cbaedbc None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:15.221860 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1f145-6cfc-4ece-9101-c9f19cbaedbc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:15.221860 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1f145-6cfc-4ece-9101-c9f19cbaedbc None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:15.223459 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 229/456] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:15.229366 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdce540-cae6-4c25-90d0-531f97e011e5 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:15.230240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdce540-cae6-4c25-90d0-531f97e011e5 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:15.230717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdce540-cae6-4c25-90d0-531f97e011e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:15.256144 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-8fdce540-cae6-4c25-90d0-531f97e011e5 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:15.258359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8fdce540-cae6-4c25-90d0-531f97e011e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:15.350443 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 228/457] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:15.669053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1f145-6cfc-4ece-9101-c9f19cbaedbc None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:15.734237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:15.746602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de531a-359a-487a-8900-0666c1f3a264 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:15.746894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de531a-359a-487a-8900-0666c1f3a264 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:15.747171 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de531a-359a-487a-8900-0666c1f3a264 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:15.748820 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 229/458] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:15.756216 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f75855-a498-4792-8935-105d16579d0c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:15.756959 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f75855-a498-4792-8935-105d16579d0c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:15.757508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f75855-a498-4792-8935-105d16579d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:15.789143 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-26f75855-a498-4792-8935-105d16579d0c None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:15.790718 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-26f75855-a498-4792-8935-105d16579d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:15.821080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1f145-6cfc-4ece-9101-c9f19cbaedbc None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:15.879151 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 230/459] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:15.880899 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf633f-3e71-431f-8364-19ea757f0fa2 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:15.881391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf633f-3e71-431f-8364-19ea757f0fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:15.881951 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf633f-3e71-431f-8364-19ea757f0fa2 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:15.883762 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 231/460] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:15.887318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d88ff7-87d7-406a-a869-e83be069900f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:15.888319 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d88ff7-87d7-406a-a869-e83be069900f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:15.888800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d88ff7-87d7-406a-a869-e83be069900f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:15.889196 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08915b45-06a1-4e34-81ca-8c66b704de9c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:15.889379 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d88ff7-87d7-406a-a869-e83be069900f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:15.889783 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08915b45-06a1-4e34-81ca-8c66b704de9c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:15.890225 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08915b45-06a1-4e34-81ca-8c66b704de9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:15.913815 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d88ff7-87d7-406a-a869-e83be069900f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:15.915712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d88ff7-87d7-406a-a869-e83be069900f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:15.920081 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-08915b45-06a1-4e34-81ca-8c66b704de9c None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:15.921805 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-08915b45-06a1-4e34-81ca-8c66b704de9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:15.945479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-80d88ff7-87d7-406a-a869-e83be069900f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:15.945479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:15.945479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:15.945479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:15.945479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:15.945479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:15.945479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:15.945479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:15.945479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:15.945479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:15.945479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:15.945479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:15.945479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:15.945479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:15.945479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:15.945479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:15.946636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:15.946636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:15.946636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:15.946636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:15.946636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:15.946636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:15.946636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:15.946636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:15.946636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:15.946636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:15.946636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:15.946636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:15.946636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:15.946636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:15.946636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:15.946636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:15.946636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:15.947704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:15.947704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:15.947704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:15.949477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 230/461] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:15.955255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d88ff7-87d7-406a-a869-e83be069900f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:15.964446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:16.054726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 232/462] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:42:16.056431 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bd557-d24d-4a9d-b8fe-d23a9dcac0ff None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:16.056902 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bd557-d24d-4a9d-b8fe-d23a9dcac0ff None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:16.057345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bd557-d24d-4a9d-b8fe-d23a9dcac0ff None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:16.062871 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 233/463] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:16.064602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335575cd-3e86-4b7b-98d6-f56f90f20ddf None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:16.065066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335575cd-3e86-4b7b-98d6-f56f90f20ddf None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:16.065518 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335575cd-3e86-4b7b-98d6-f56f90f20ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:16.084414 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-335575cd-3e86-4b7b-98d6-f56f90f20ddf None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:16.086043 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-335575cd-3e86-4b7b-98d6-f56f90f20ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:16.095000 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:16.108419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d88ff7-87d7-406a-a869-e83be069900f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:16.148911 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88bda3-7aee-4262-8aeb-e295a0998147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:16.150125 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88bda3-7aee-4262-8aeb-e295a0998147 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:16.150411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88bda3-7aee-4262-8aeb-e295a0998147 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:16.150685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88bda3-7aee-4262-8aeb-e295a0998147 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:16.151337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88bda3-7aee-4262-8aeb-e295a0998147 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:16.151642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88bda3-7aee-4262-8aeb-e295a0998147 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:16.153064 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88bda3-7aee-4262-8aeb-e295a0998147 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:16.199770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 231/464] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:16.206048 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88bda3-7aee-4262-8aeb-e295a0998147 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:16.228952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 234/465] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:42:16.230507 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335575cd-3e86-4b7b-98d6-f56f90f20ddf None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:16.328418 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:16.352901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335575cd-3e86-4b7b-98d6-f56f90f20ddf None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:16.398613 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df89aa4-5c09-4cf0-8b61-c246da5cb229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:16.399653 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df89aa4-5c09-4cf0-8b61-c246da5cb229 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:16.399760 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88bda3-7aee-4262-8aeb-e295a0998147 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:16.400604 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df89aa4-5c09-4cf0-8b61-c246da5cb229 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:16.401075 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df89aa4-5c09-4cf0-8b61-c246da5cb229 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:16.416017 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df89aa4-5c09-4cf0-8b61-c246da5cb229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:16.418296 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df89aa4-5c09-4cf0-8b61-c246da5cb229 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:16.437716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-2df89aa4-5c09-4cf0-8b61-c246da5cb229 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:16.437716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:16.437716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:16.437716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:16.437716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:16.437716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:16.437716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:16.437716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:16.437716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:16.437716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:16.437716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:16.437716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:16.437716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:16.437716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:16.437716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:16.437716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:16.438652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:16.438652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:16.438652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:16.438652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:16.438652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:16.438652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:16.438652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:16.438652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:16.438652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:16.438652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:16.438652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:16.438652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:16.438652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:16.438652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:16.438652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:16.438652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:16.438652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:16.439478 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:16.439478 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:16.439478 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:16.439478 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 235/466] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:16.439770 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df89aa4-5c09-4cf0-8b61-c246da5cb229 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:16.440800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e42d25-61ba-4d52-acc7-2950013f1940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:16.441411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e42d25-61ba-4d52-acc7-2950013f1940 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:16.441681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e42d25-61ba-4d52-acc7-2950013f1940 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:16.441944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e42d25-61ba-4d52-acc7-2950013f1940 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:16.442597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e42d25-61ba-4d52-acc7-2950013f1940 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:16.444030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e42d25-61ba-4d52-acc7-2950013f1940 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:16.471608 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 232/467] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 24 08:42:16 2022] POST /identity/v3/users => generated 246 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:42:16.476646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e42d25-61ba-4d52-acc7-2950013f1940 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:16.503260 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:16.618723 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df89aa4-5c09-4cf0-8b61-c246da5cb229 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:16.655036 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:16.662346 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bf5b8-af1a-4e62-823c-940de2b3656a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:16.663492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bf5b8-af1a-4e62-823c-940de2b3656a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:16.663776 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bf5b8-af1a-4e62-823c-940de2b3656a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:16.664051 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bf5b8-af1a-4e62-823c-940de2b3656a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:16.664891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e42d25-61ba-4d52-acc7-2950013f1940 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:16.675968 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bf5b8-af1a-4e62-823c-940de2b3656a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:16.677479 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bf5b8-af1a-4e62-823c-940de2b3656a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:16.700843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-b60bf5b8-af1a-4e62-823c-940de2b3656a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:16.700843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:16.700843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:16.700843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:16.700843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:16.700843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:16.700843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:16.700843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:16.700843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:16.700843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:16.700843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:16.700843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:16.700843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:16.700843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:16.700843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:16.700843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:16.701801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:16.701801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:16.701801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:16.701801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:16.701801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:16.701801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:16.701801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:16.701801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:16.701801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:16.701801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:16.701801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:16.701801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:16.701801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:16.701801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:16.701801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:16.701801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:16.701801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:16.702643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:16.702643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:16.702643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:16.702643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 236/468] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:16.702866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bf5b8-af1a-4e62-823c-940de2b3656a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:16.722363 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e42e8b-3e63-438c-9417-c5e8607d7b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:16.722916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e42e8b-3e63-438c-9417-c5e8607d7b05 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:16.723194 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e42e8b-3e63-438c-9417-c5e8607d7b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:16.723466 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e42e8b-3e63-438c-9417-c5e8607d7b05 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:16.724865 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e42e8b-3e63-438c-9417-c5e8607d7b05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:16.725184 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e42e8b-3e63-438c-9417-c5e8607d7b05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:16.727403 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e42e8b-3e63-438c-9417-c5e8607d7b05 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:16.742373 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 233/469] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:16.743280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e42e8b-3e63-438c-9417-c5e8607d7b05 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:16.819760 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:16.828885 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bf5b8-af1a-4e62-823c-940de2b3656a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:16.860629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e42e8b-3e63-438c-9417-c5e8607d7b05 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:16.868529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b070e2-b293-43bc-819e-e1cc9f96cce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:16.869174 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b070e2-b293-43bc-819e-e1cc9f96cce8 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:16.869455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b070e2-b293-43bc-819e-e1cc9f96cce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:16.869730 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b070e2-b293-43bc-819e-e1cc9f96cce8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:16.881401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b070e2-b293-43bc-819e-e1cc9f96cce8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:16.884054 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b070e2-b293-43bc-819e-e1cc9f96cce8 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:16.898554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f12ea4-3782-4914-ba97-84555c851079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:16.899167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f12ea4-3782-4914-ba97-84555c851079 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:16.899448 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f12ea4-3782-4914-ba97-84555c851079 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:16.899720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f12ea4-3782-4914-ba97-84555c851079 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:16.899817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-73b070e2-b293-43bc-819e-e1cc9f96cce8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:16.899817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:16.899817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:16.899817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:16.899817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:16.899817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:16.899817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:16.899817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:16.899817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:16.899817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:16.899817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:16.899817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:16.899817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:16.899817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:16.899817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:16.899817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:16.900819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:16.900819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:16.900819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:16.900819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:16.900819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:16.900819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:16.900819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:16.900819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:16.900819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:16.900819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:16.900819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:16.900819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:16.900819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:16.900819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:16.900819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:16.900819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:16.900819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:16.909503 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:16.909503 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:16.909503 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:16.909670 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f12ea4-3782-4914-ba97-84555c851079 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:16.909670 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f12ea4-3782-4914-ba97-84555c851079 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:16.909809 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 237/470] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:16.909876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f12ea4-3782-4914-ba97-84555c851079 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:16.909942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9c9be-2c6b-49cc-8a88-9aa64d60337d None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:16.909942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9c9be-2c6b-49cc-8a88-9aa64d60337d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:16.909942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9c9be-2c6b-49cc-8a88-9aa64d60337d None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:16.909942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 238/471] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:16.909942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9c9be-2c6b-49cc-8a88-9aa64d60337d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:16.922338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 234/472] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:16.923123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f12ea4-3782-4914-ba97-84555c851079 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:16.954015 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:17.026524 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9c9be-2c6b-49cc-8a88-9aa64d60337d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:17.045704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f12ea4-3782-4914-ba97-84555c851079 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:17.070719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a8397-a7f4-4446-bc94-860b389709f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:17.071601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a8397-a7f4-4446-bc94-860b389709f3 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:17.071903 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a8397-a7f4-4446-bc94-860b389709f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:17.072331 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a8397-a7f4-4446-bc94-860b389709f3 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:17.072996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a8397-a7f4-4446-bc94-860b389709f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:17.074456 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a8397-a7f4-4446-bc94-860b389709f3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:17.076740 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:17.087550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf9c30-0c3a-44bc-a002-d354247cd2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:17.088249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf9c30-0c3a-44bc-a002-d354247cd2e3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:17.088712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf9c30-0c3a-44bc-a002-d354247cd2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:17.089117 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf9c30-0c3a-44bc-a002-d354247cd2e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:17.090129 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf9c30-0c3a-44bc-a002-d354247cd2e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:17.090517 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf9c30-0c3a-44bc-a002-d354247cd2e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:17.092294 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf9c30-0c3a-44bc-a002-d354247cd2e3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:17.098804 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 239/473] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 24 08:42:16 2022] POST /identity/v3/users => generated 248 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:42:17.105728 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080899c-5824-41a9-ac31-06e515985f7d None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:17.105952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080899c-5824-41a9-ac31-06e515985f7d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:17.107318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080899c-5824-41a9-ac31-06e515985f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:17.110726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 235/474] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:17.112184 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf9c30-0c3a-44bc-a002-d354247cd2e3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:17.137909 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9080899c-5824-41a9-ac31-06e515985f7d None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:17.138996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9080899c-5824-41a9-ac31-06e515985f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:17.233091 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 240/475] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:17.233091 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080899c-5824-41a9-ac31-06e515985f7d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:17.273661 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:17.285325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf9c30-0c3a-44bc-a002-d354247cd2e3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:17.327929 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9aca2-7c35-48a3-8ad5-0c459a611505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:17.328786 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9aca2-7c35-48a3-8ad5-0c459a611505 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:17.329203 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9aca2-7c35-48a3-8ad5-0c459a611505 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:17.329570 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9aca2-7c35-48a3-8ad5-0c459a611505 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:17.330329 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9aca2-7c35-48a3-8ad5-0c459a611505 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:17.331902 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9aca2-7c35-48a3-8ad5-0c459a611505 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:17.363676 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 236/476] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 24 08:42:17 2022] POST /identity/v3/users => generated 248 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:42:17.406491 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080899c-5824-41a9-ac31-06e515985f7d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:17.457257 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b043d-e81b-45c2-ae12-a5c43ec50bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:17.458308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b043d-e81b-45c2-ae12-a5c43ec50bf4 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:17.458817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b043d-e81b-45c2-ae12-a5c43ec50bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:17.459359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b043d-e81b-45c2-ae12-a5c43ec50bf4 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:17.462022 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b043d-e81b-45c2-ae12-a5c43ec50bf4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:17.463780 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b043d-e81b-45c2-ae12-a5c43ec50bf4 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:17.499125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 241/477] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 24 08:42:17 2022] POST /identity/v3/users => generated 249 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:42:17.567257 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9aca2-7c35-48a3-8ad5-0c459a611505 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:17.630025 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:17.698336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9aca2-7c35-48a3-8ad5-0c459a611505 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:17.748366 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5497e-b085-4e4d-a846-3450c449c958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:17.749041 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5497e-b085-4e4d-a846-3450c449c958 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:17.749322 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5497e-b085-4e4d-a846-3450c449c958 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:17.749597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5497e-b085-4e4d-a846-3450c449c958 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:17.750283 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5497e-b085-4e4d-a846-3450c449c958 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:17.751774 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5497e-b085-4e4d-a846-3450c449c958 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:17.762774 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.sql.core [None req-d9f5497e-b085-4e4d-a846-3450c449c958 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 08:42:17.762774 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 08:42:17.762774 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [parameters: {'id': '0b81c5be32d24eca82391e6d2eecc9a9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 08:42:17.762774 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81475) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 08:42:17.781101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d9f5497e-b085-4e4d-a846-3450c449c958 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 08:42:17.781101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:17.781101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 08:42:17.781101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 08:42:17.781101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 08:42:17.781101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 08:42:17.781101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 08:42:17.781101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 08:42:17.781101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 08:42:17.781101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application conn.query(q) Aug 24 08:42:17.781101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 08:42:17.781101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 08:42:17.781101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 08:42:17.781101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result.read() Aug 24 08:42:17.781101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 08:42:17.782278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 08:42:17.782278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 08:42:17.782278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 08:42:17.782278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 08:42:17.782278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 08:42:17.782278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 08:42:17.782278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 08:42:17.782278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 08:42:17.782278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:17.782278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 08:42:17.782278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:17.782278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:17.782278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 08:42:17.782278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 08:42:17.782278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 08:42:17.782278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 08:42:17.782278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 08:42:17.782278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application next(self.gen) Aug 24 08:42:17.783506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 08:42:17.783506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application yield resource Aug 24 08:42:17.783506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 08:42:17.783506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application next(self.gen) Aug 24 08:42:17.783506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 08:42:17.783506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 08:42:17.783506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 08:42:17.783506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 08:42:17.783506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 08:42:17.783506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise self.value Aug 24 08:42:17.783506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 08:42:17.783506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 08:42:17.783506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 08:42:17.783506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application session.commit() Aug 24 08:42:17.783506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 08:42:17.783506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 08:42:17.783506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 08:42:17.784605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 08:42:17.784605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 08:42:17.784605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.session.flush() Aug 24 08:42:17.784605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 08:42:17.784605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 08:42:17.784605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 08:42:17.784605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 08:42:17.784605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 08:42:17.784605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application compat.raise_( Aug 24 08:42:17.784605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 08:42:17.784605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception Aug 24 08:42:17.784605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 08:42:17.784605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 08:42:17.784605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 08:42:17.784605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 08:42:17.784605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 08:42:17.784605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 08:42:17.786247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 08:42:17.786247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 08:42:17.786247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 08:42:17.786247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 08:42:17.786247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 08:42:17.786247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 08:42:17.786247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 08:42:17.786247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 08:42:17.786247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 08:42:17.786247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 08:42:17.786247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 08:42:17.786247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 08:42:17.786247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 08:42:17.786247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 08:42:17.786247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 08:42:17.786247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception Aug 24 08:42:17.789014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 08:42:17.789014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 08:42:17.789014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 08:42:17.789014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 08:42:17.789014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 08:42:17.789014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 08:42:17.789014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 08:42:17.789014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application conn.query(q) Aug 24 08:42:17.789014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 08:42:17.789014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 08:42:17.789014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 08:42:17.789014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result.read() Aug 24 08:42:17.789014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 08:42:17.789014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 08:42:17.789014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 08:42:17.789014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 08:42:17.791093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 08:42:17.791093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 08:42:17.791093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 08:42:17.791093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 08:42:17.791093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 08:42:17.791093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 08:42:17.791093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [parameters: {'id': '0b81c5be32d24eca82391e6d2eecc9a9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 08:42:17.791093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 08:42:17.791093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:17.791093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:42:17.791093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:17.791093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:17.791093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:17.791093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:17.791093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:17.791093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:17.793419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:17.793419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:17.793419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:17.793419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:17.793419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:17.793419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:17.793419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 08:42:17.793419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 08:42:17.793419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:17.793419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:17.793419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 08:42:17.793419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 08:42:17.793419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 08:42:17.793419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 08:42:17.793419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 08:42:17.793419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:17.795347 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 237/478] 10.222.0.67 () {60 vars in 1131 bytes} [Wed Aug 24 08:42:17 2022] POST /identity/v3/roles => generated 144 bytes in 220 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:42:17.798183 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b043d-e81b-45c2-ae12-a5c43ec50bf4 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:17.935990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:17.968126 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b043d-e81b-45c2-ae12-a5c43ec50bf4 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:18.018315 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6cd1e-5ae0-45cd-94d2-e2c868eee7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:18.018971 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6cd1e-5ae0-45cd-94d2-e2c868eee7ea None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:18.019252 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6cd1e-5ae0-45cd-94d2-e2c868eee7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:18.019525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6cd1e-5ae0-45cd-94d2-e2c868eee7ea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:18.035045 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6cd1e-5ae0-45cd-94d2-e2c868eee7ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:18.041197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-a0f6cd1e-5ae0-45cd-94d2-e2c868eee7ea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:42:18.041197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:18.041197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:18.041197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:18.041197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:18.041197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:18.041197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:18.041197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:18.041197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:18.041197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:18.041197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:18.041197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:18.041197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:18.041197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:18.041197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:18.041197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:18.043286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:18.043286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:18.043286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:18.043286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:18.043286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:18.043286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:18.043286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:18.043286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:18.043286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:18.043286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:18.043286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:18.043286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:18.043286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:18.043286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:18.043286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:18.043286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:18.045250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:18.045250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:18.045250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:18.045250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:18.045250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:18.045250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:18.045250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:18.045250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:18.045250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:18.045250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:18.045250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 08:42:18.045250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:18.045250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 242/479] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:42:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:18.048480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5497e-b085-4e4d-a846-3450c449c958 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:18.322473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5497e-b085-4e4d-a846-3450c449c958 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:18.349496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:18.387555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b456e06-b6cd-4739-9cc1-58492afc1207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:18.388649 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b456e06-b6cd-4739-9cc1-58492afc1207 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:18.389421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b456e06-b6cd-4739-9cc1-58492afc1207 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:18.390352 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b456e06-b6cd-4739-9cc1-58492afc1207 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:18.391744 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b456e06-b6cd-4739-9cc1-58492afc1207 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:18.392461 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b456e06-b6cd-4739-9cc1-58492afc1207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:18.395119 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b456e06-b6cd-4739-9cc1-58492afc1207 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:18.411150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 238/480] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:18.959613 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f743552-4c06-4114-bc09-2a82eb3af16c None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:18.960527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f743552-4c06-4114-bc09-2a82eb3af16c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:18.960980 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f743552-4c06-4114-bc09-2a82eb3af16c None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:18.962859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 243/481] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:18.970829 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32639b5c-169d-481c-a5d3-71926f128138 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:18.971726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32639b5c-169d-481c-a5d3-71926f128138 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:18.972221 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32639b5c-169d-481c-a5d3-71926f128138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:18.993957 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-32639b5c-169d-481c-a5d3-71926f128138 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:18.995576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-32639b5c-169d-481c-a5d3-71926f128138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:19.087634 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 239/482] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:42:19.424124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f743552-4c06-4114-bc09-2a82eb3af16c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:19.557401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f743552-4c06-4114-bc09-2a82eb3af16c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:19.606422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552bd2a8-28d9-4732-9dc3-cd674997f83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:19.607143 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bd2a8-28d9-4732-9dc3-cd674997f83a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:19.607415 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bd2a8-28d9-4732-9dc3-cd674997f83a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:19.607688 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bd2a8-28d9-4732-9dc3-cd674997f83a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:19.620937 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:19.633335 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552bd2a8-28d9-4732-9dc3-cd674997f83a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:19.639666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-552bd2a8-28d9-4732-9dc3-cd674997f83a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:19.639666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:19.639666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:19.639666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:19.639666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:19.639666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:19.639666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:19.639666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:19.639666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:19.639666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:19.639666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:19.639666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:19.639666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:19.639666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:19.639666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:19.639666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:19.641389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:19.641389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:19.641389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:19.641389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:19.641389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:19.641389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:19.641389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:19.641389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:19.641389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:19.641389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:19.641389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:19.641389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:19.641389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:19.641389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:19.641389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:19.641389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:19.643099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:19.643099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:19.643099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:19.643099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:19.643099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:19.643099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:19.643099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:19.643099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:19.643099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:19.643099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:19.643099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:19.643099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:19.643099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 244/483] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:19.650094 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32639b5c-169d-481c-a5d3-71926f128138 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:19.718177 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e089d-f920-4226-b44b-f3f402bdd015 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:19.718177 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e089d-f920-4226-b44b-f3f402bdd015 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:19.718177 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e089d-f920-4226-b44b-f3f402bdd015 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:19.718657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 245/484] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:19.735144 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8b460-ef72-4b7d-b631-5e70b5cc5f36 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:19.735554 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8b460-ef72-4b7d-b631-5e70b5cc5f36 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:19.735834 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8b460-ef72-4b7d-b631-5e70b5cc5f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:19.772638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-8de8b460-ef72-4b7d-b631-5e70b5cc5f36 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:19.774364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8de8b460-ef72-4b7d-b631-5e70b5cc5f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:19.871654 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:19.876182 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32639b5c-169d-481c-a5d3-71926f128138 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:19.921856 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 246/485] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:42:19.946589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149421b8-2bc6-4f12-9d99-a9dfaf6ae511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:19.947646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149421b8-2bc6-4f12-9d99-a9dfaf6ae511 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:19.948141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149421b8-2bc6-4f12-9d99-a9dfaf6ae511 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:19.948606 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149421b8-2bc6-4f12-9d99-a9dfaf6ae511 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:19.949475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149421b8-2bc6-4f12-9d99-a9dfaf6ae511 None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:19.949992 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149421b8-2bc6-4f12-9d99-a9dfaf6ae511 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:19.951627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149421b8-2bc6-4f12-9d99-a9dfaf6ae511 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:19.962116 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 240/486] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:42:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:19.968153 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8b460-ef72-4b7d-b631-5e70b5cc5f36 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:20.050267 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d57b7-bf62-472f-955e-d6285211b898 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:20.051221 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d57b7-bf62-472f-955e-d6285211b898 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:20.051704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d57b7-bf62-472f-955e-d6285211b898 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:20.053550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 241/487] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:20.060755 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98075e7c-bee6-4196-ac5e-26fbae4dac8e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:20.061584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98075e7c-bee6-4196-ac5e-26fbae4dac8e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:20.062577 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98075e7c-bee6-4196-ac5e-26fbae4dac8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:20.091889 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-98075e7c-bee6-4196-ac5e-26fbae4dac8e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:20.095121 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-98075e7c-bee6-4196-ac5e-26fbae4dac8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:20.139024 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8b460-ef72-4b7d-b631-5e70b5cc5f36 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:20.169950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:20.180406 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222d765-b424-4be6-af10-b888d5dbc03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:20.181417 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222d765-b424-4be6-af10-b888d5dbc03e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:20.181938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222d765-b424-4be6-af10-b888d5dbc03e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:20.182402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222d765-b424-4be6-af10-b888d5dbc03e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:20.191846 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 242/488] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:42:20.191846 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c47cf-41a3-4f17-a37a-f50f36c835e6 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:20.191846 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c47cf-41a3-4f17-a37a-f50f36c835e6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:20.191846 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c47cf-41a3-4f17-a37a-f50f36c835e6 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:20.192281 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 243/489] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:20.194105 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440c319-37be-4034-bb9e-c3995b89af06 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:20.194419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440c319-37be-4034-bb9e-c3995b89af06 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:20.194696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440c319-37be-4034-bb9e-c3995b89af06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:20.207402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222d765-b424-4be6-af10-b888d5dbc03e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:20.209342 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222d765-b424-4be6-af10-b888d5dbc03e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:20.216030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a440c319-37be-4034-bb9e-c3995b89af06 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:20.217257 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a440c319-37be-4034-bb9e-c3995b89af06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:20.239030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-3222d765-b424-4be6-af10-b888d5dbc03e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:20.239030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:20.239030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:20.239030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:20.239030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:20.239030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:20.239030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:20.239030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:20.239030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:20.239030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:20.239030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:20.239030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:20.239030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:20.239030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:20.239030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:20.239030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:20.239986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:20.239986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:20.239986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:20.239986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:20.239986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:20.239986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:20.239986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:20.239986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:20.239986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:20.239986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:20.239986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:20.239986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:20.239986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:20.239986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:20.239986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:20.239986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:20.239986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:20.240855 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:20.240855 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:20.240855 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:20.243836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 247/490] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:20.249781 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222d765-b424-4be6-af10-b888d5dbc03e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:20.309629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 244/491] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:20.420234 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222d765-b424-4be6-af10-b888d5dbc03e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:20.425723 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440c319-37be-4034-bb9e-c3995b89af06 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:20.463837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6c7ed-ccd3-4701-b197-d8da59acb4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:20.464875 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6c7ed-ccd3-4701-b197-d8da59acb4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:20.465377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6c7ed-ccd3-4701-b197-d8da59acb4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:20.465846 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6c7ed-ccd3-4701-b197-d8da59acb4a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:20.466692 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6c7ed-ccd3-4701-b197-d8da59acb4a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:20.467192 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6c7ed-ccd3-4701-b197-d8da59acb4a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:20.468818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6c7ed-ccd3-4701-b197-d8da59acb4a9 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:20.472219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:20.483837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 248/492] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:20.489051 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6c7ed-ccd3-4701-b197-d8da59acb4a9 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:20.538634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:20.623380 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440c319-37be-4034-bb9e-c3995b89af06 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:20.657580 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6c7ed-ccd3-4701-b197-d8da59acb4a9 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:20.662051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83917c9d-21ff-4742-883c-7007c989d511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:20.663493 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83917c9d-21ff-4742-883c-7007c989d511 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:20.663982 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83917c9d-21ff-4742-883c-7007c989d511 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:20.664563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83917c9d-21ff-4742-883c-7007c989d511 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:20.675761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83917c9d-21ff-4742-883c-7007c989d511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:20.681530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-83917c9d-21ff-4742-883c-7007c989d511 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:20.681530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:20.681530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:20.681530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:20.681530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:20.681530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:20.681530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:20.681530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:20.681530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:20.681530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:20.681530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:20.681530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:20.681530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:20.681530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:20.681530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:20.681530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:20.682408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:20.682408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:20.682408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:20.682408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:20.682408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:20.682408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:20.682408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:20.682408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:20.682408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:20.682408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:20.682408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:20.682408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:20.682408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:20.682408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:20.682408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:20.682408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:20.683202 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:20.683202 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:20.683202 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:20.683202 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:20.683202 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:20.683202 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:20.683202 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:20.683202 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:20.683202 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:20.683202 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:20.683202 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:20.683202 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:20.685884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 245/493] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:42:20.686952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83917c9d-21ff-4742-883c-7007c989d511 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:20.722620 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772933fd-a893-4d9c-a559-c09e43ae1212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:20.723870 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772933fd-a893-4d9c-a559-c09e43ae1212 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:20.724714 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772933fd-a893-4d9c-a559-c09e43ae1212 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:20.725086 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772933fd-a893-4d9c-a559-c09e43ae1212 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:20.739514 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772933fd-a893-4d9c-a559-c09e43ae1212 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:20.742302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772933fd-a893-4d9c-a559-c09e43ae1212 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:20.768737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-772933fd-a893-4d9c-a559-c09e43ae1212 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:42:20.768737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:20.768737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:20.768737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:20.768737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:20.768737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:20.768737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:20.768737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:20.768737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:20.768737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:20.768737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:20.768737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:20.768737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:20.768737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:20.768737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:20.768737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:20.769758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:20.769758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:20.769758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:20.769758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:20.769758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:20.769758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:20.769758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:20.769758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:20.769758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:20.769758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:20.769758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:20.769758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:20.769758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:20.769758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:20.769758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:20.769758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:20.769758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:20.769758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:20.770654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:20.770654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:20.770654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:20.770654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:20.770654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:20.770654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:20.770654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:20.770654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:20.770654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:42:20.770654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:20.771182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 249/494] 10.222.0.67 () {56 vars in 1093 bytes} [Wed Aug 24 08:42:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:42:20.772044 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772933fd-a893-4d9c-a559-c09e43ae1212 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:20.793878 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:20.815796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83917c9d-21ff-4742-883c-7007c989d511 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:20.860082 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a059635-d943-4d5a-a197-0017126e5cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:20.860746 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a059635-d943-4d5a-a197-0017126e5cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:20.861072 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a059635-d943-4d5a-a197-0017126e5cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:20.861315 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a059635-d943-4d5a-a197-0017126e5cb8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:20.874529 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a059635-d943-4d5a-a197-0017126e5cb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:20.879423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-7a059635-d943-4d5a-a197-0017126e5cb8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:20.879423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:20.879423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:20.879423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:20.879423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:20.879423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:20.879423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:20.879423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:20.879423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:20.879423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:20.879423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:20.879423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:20.879423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:20.879423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:20.879423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:20.879423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:20.880710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:20.880710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:20.880710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:20.880710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:20.880710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:20.880710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:20.880710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:20.880710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:20.880710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:20.880710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:20.880710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:20.880710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:20.880710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:20.880710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:20.880710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:20.880710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:20.881686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:20.881686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:20.881686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:20.881686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:20.881686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:20.881686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:20.881686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:20.881686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:20.881686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:20.881686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:20.881686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:20.881686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:20.881686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 246/495] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:20.881686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a059635-d943-4d5a-a197-0017126e5cb8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:20.910880 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:20.914334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772933fd-a893-4d9c-a559-c09e43ae1212 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:20.958330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c283f5-6713-4634-b438-fec625783160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:20.959159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c283f5-6713-4634-b438-fec625783160 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:20.960328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c283f5-6713-4634-b438-fec625783160 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:20.960503 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c283f5-6713-4634-b438-fec625783160 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:20.961244 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c283f5-6713-4634-b438-fec625783160 None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:20.961553 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c283f5-6713-4634-b438-fec625783160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:20.963138 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c283f5-6713-4634-b438-fec625783160 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:20.972297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 250/496] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:42:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:20.973010 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c283f5-6713-4634-b438-fec625783160 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:21.021615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a059635-d943-4d5a-a197-0017126e5cb8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:21.064333 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdaed35-6134-4624-abdc-ae8600a92c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:21.064972 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdaed35-6134-4624-abdc-ae8600a92c89 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:21.065239 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdaed35-6134-4624-abdc-ae8600a92c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:21.065503 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdaed35-6134-4624-abdc-ae8600a92c89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:21.078568 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdaed35-6134-4624-abdc-ae8600a92c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:21.083449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ecdaed35-6134-4624-abdc-ae8600a92c89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:21.083449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:21.083449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:21.083449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:21.083449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:21.083449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:21.083449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:21.083449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:21.083449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:21.083449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:21.083449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:21.083449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:21.083449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:21.083449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:21.083449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:21.083449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:21.085258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:21.085258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:21.085258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:21.085258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:21.085258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:21.085258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:21.085258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:21.085258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:21.085258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:21.085258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:21.085258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:21.085258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:21.085258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:21.085258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:21.085258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:21.085258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:21.086904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:21.086904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:21.086904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:21.086904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:21.086904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:21.086904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:21.086904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:21.086904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:21.086904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:21.086904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:21.086904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:21.086904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:21.086904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 247/497] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:21.090770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdaed35-6134-4624-abdc-ae8600a92c89 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:21.095335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:21.143998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c283f5-6713-4634-b438-fec625783160 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:21.206365 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b928a-2a0a-45a6-a1e8-dd523c08bac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:21.206973 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b928a-2a0a-45a6-a1e8-dd523c08bac1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:21.207255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b928a-2a0a-45a6-a1e8-dd523c08bac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:21.207521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b928a-2a0a-45a6-a1e8-dd523c08bac1 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:21.208306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b928a-2a0a-45a6-a1e8-dd523c08bac1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:21.208614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b928a-2a0a-45a6-a1e8-dd523c08bac1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:21.210647 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b928a-2a0a-45a6-a1e8-dd523c08bac1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:21.214699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdaed35-6134-4624-abdc-ae8600a92c89 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:21.259595 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 251/498] 10.222.0.67 () {56 vars in 1133 bytes} [Wed Aug 24 08:42:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:21.259595 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b928a-2a0a-45a6-a1e8-dd523c08bac1 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:21.270964 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d358bce-a1de-4768-b4a1-2d6868419249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:21.271586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d358bce-a1de-4768-b4a1-2d6868419249 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:21.271870 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d358bce-a1de-4768-b4a1-2d6868419249 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:21.272221 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d358bce-a1de-4768-b4a1-2d6868419249 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:21.273652 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d358bce-a1de-4768-b4a1-2d6868419249 None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:21.273955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d358bce-a1de-4768-b4a1-2d6868419249 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:21.275469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d358bce-a1de-4768-b4a1-2d6868419249 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:21.289523 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 248/499] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:42:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:21.290385 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d358bce-a1de-4768-b4a1-2d6868419249 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:21.295553 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:21.368229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:21.392270 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b928a-2a0a-45a6-a1e8-dd523c08bac1 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:21.414228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d358bce-a1de-4768-b4a1-2d6868419249 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:21.428011 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867393fe-5e60-4124-9f05-2b0d42ae565b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:21.429332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867393fe-5e60-4124-9f05-2b0d42ae565b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:21.429719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867393fe-5e60-4124-9f05-2b0d42ae565b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:21.430025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867393fe-5e60-4124-9f05-2b0d42ae565b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:21.442142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867393fe-5e60-4124-9f05-2b0d42ae565b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:21.443695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867393fe-5e60-4124-9f05-2b0d42ae565b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:21.453547 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e3705-626c-4e72-9b26-076e9f821435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:21.454178 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e3705-626c-4e72-9b26-076e9f821435 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:21.454452 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e3705-626c-4e72-9b26-076e9f821435 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:21.454722 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e3705-626c-4e72-9b26-076e9f821435 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:21.455415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e3705-626c-4e72-9b26-076e9f821435 None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:21.455717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e3705-626c-4e72-9b26-076e9f821435 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:21.457188 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e3705-626c-4e72-9b26-076e9f821435 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:21.460960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-867393fe-5e60-4124-9f05-2b0d42ae565b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:21.460960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:21.460960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:21.460960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:21.460960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:21.460960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:21.460960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:21.460960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:21.460960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:21.460960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:21.460960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:21.460960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:21.460960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:21.460960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:21.460960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:21.460960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:21.461918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:21.461918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:21.461918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:21.461918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:21.461918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:21.461918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:21.461918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:21.461918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:21.461918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:21.461918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:21.461918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:21.461918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:21.461918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:21.461918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:21.461918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:21.461918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:21.461918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:21.462793 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:21.462793 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:21.462793 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:21.462793 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 252/500] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:21.463017 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867393fe-5e60-4124-9f05-2b0d42ae565b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:21.469088 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 249/501] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:42:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:21.469500 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e3705-626c-4e72-9b26-076e9f821435 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:21.547233 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:21.580479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e3705-626c-4e72-9b26-076e9f821435 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:21.580717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867393fe-5e60-4124-9f05-2b0d42ae565b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:21.609433 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:21.619627 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22733977-6e80-4bb2-8faa-1ae5a103c77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:21.620413 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22733977-6e80-4bb2-8faa-1ae5a103c77a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:21.620789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22733977-6e80-4bb2-8faa-1ae5a103c77a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:21.621100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328fa22-6e29-4e6d-b748-9a8139c62c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:21.621219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22733977-6e80-4bb2-8faa-1ae5a103c77a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:21.621706 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328fa22-6e29-4e6d-b748-9a8139c62c61 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:21.621970 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328fa22-6e29-4e6d-b748-9a8139c62c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:21.622236 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328fa22-6e29-4e6d-b748-9a8139c62c61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:21.633741 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328fa22-6e29-4e6d-b748-9a8139c62c61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:21.635213 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328fa22-6e29-4e6d-b748-9a8139c62c61 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:21.650832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-e328fa22-6e29-4e6d-b748-9a8139c62c61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:21.650832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:21.650832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:21.650832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:21.650832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:21.650832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:21.650832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:21.650832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:21.650832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:21.650832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:21.650832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:21.650832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:21.650832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:21.650832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:21.650832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:21.650832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:21.651800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:21.651800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:21.651800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:21.651800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:21.651800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:21.651800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:21.651800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:21.651800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:21.651800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:21.651800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:21.651800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:21.651800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:21.651800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:21.651800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:21.651800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:21.651800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:21.651800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:21.653317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:21.653317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:21.653317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:21.653317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 250/502] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:21.653317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2d222-d922-40c5-b8de-abf965aa7a2c None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:21.653635 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2d222-d922-40c5-b8de-abf965aa7a2c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:21.653635 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2d222-d922-40c5-b8de-abf965aa7a2c None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:21.654931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 251/503] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:21.655612 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2d222-d922-40c5-b8de-abf965aa7a2c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:21.714574 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22733977-6e80-4bb2-8faa-1ae5a103c77a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:21.716375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22733977-6e80-4bb2-8faa-1ae5a103c77a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:21.763647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2d222-d922-40c5-b8de-abf965aa7a2c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:21.789550 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:21.801619 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e4932-8a9a-4bb9-9e68-61965292a510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:21.802228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e4932-8a9a-4bb9-9e68-61965292a510 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:21.802505 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e4932-8a9a-4bb9-9e68-61965292a510 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:21.802772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e4932-8a9a-4bb9-9e68-61965292a510 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:21.803454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e4932-8a9a-4bb9-9e68-61965292a510 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:21.803831 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e4932-8a9a-4bb9-9e68-61965292a510 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:21.805562 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e4932-8a9a-4bb9-9e68-61965292a510 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:21.817594 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 252/504] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:21.818422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e4932-8a9a-4bb9-9e68-61965292a510 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:21.827524 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 253/505] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:42:21.827524 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22733977-6e80-4bb2-8faa-1ae5a103c77a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:21.864785 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:21.926881 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e4932-8a9a-4bb9-9e68-61965292a510 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:21.931057 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22733977-6e80-4bb2-8faa-1ae5a103c77a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:21.967790 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787dfc9-883c-45bf-b9e0-f7aedb755e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:21.968341 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787dfc9-883c-45bf-b9e0-f7aedb755e39 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:21.968616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787dfc9-883c-45bf-b9e0-f7aedb755e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:21.968878 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787dfc9-883c-45bf-b9e0-f7aedb755e39 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:21.969516 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787dfc9-883c-45bf-b9e0-f7aedb755e39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:21.969811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787dfc9-883c-45bf-b9e0-f7aedb755e39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:21.971186 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787dfc9-883c-45bf-b9e0-f7aedb755e39 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:21.972652 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf35e3-7e95-4603-a0c1-978d62d4d191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:21.973956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf35e3-7e95-4603-a0c1-978d62d4d191 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:21.974429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf35e3-7e95-4603-a0c1-978d62d4d191 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:21.974888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf35e3-7e95-4603-a0c1-978d62d4d191 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:21.983520 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 254/506] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:21.984810 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512821a8-49c6-406d-9ccf-3dab0937c0d7 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:21.985099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512821a8-49c6-406d-9ccf-3dab0937c0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:21.985376 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512821a8-49c6-406d-9ccf-3dab0937c0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:21.988646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf35e3-7e95-4603-a0c1-978d62d4d191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:21.990115 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf35e3-7e95-4603-a0c1-978d62d4d191 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:22.005506 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-512821a8-49c6-406d-9ccf-3dab0937c0d7 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:22.005960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-57bf35e3-7e95-4603-a0c1-978d62d4d191 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:22.005960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:22.005960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:22.005960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:22.005960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:22.005960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:22.005960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:22.005960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:22.005960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:22.005960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:22.005960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:22.005960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:22.005960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:22.005960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:22.005960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:22.005960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:22.007016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:22.007016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:22.007016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:22.007016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:22.007016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:22.007016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:22.007016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:22.007016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:22.007016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:22.007016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:22.007016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:22.007016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:22.007016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:22.007016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:22.007016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:22.007016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:22.007016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:22.007860 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:22.007860 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:22.007860 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:22.008039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-512821a8-49c6-406d-9ccf-3dab0937c0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:22.008467 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 253/507] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:22.008467 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf35e3-7e95-4603-a0c1-978d62d4d191 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:22.039429 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:22.109666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 255/508] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:22.110652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512821a8-49c6-406d-9ccf-3dab0937c0d7 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:22.155157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:22.170323 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf35e3-7e95-4603-a0c1-978d62d4d191 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:22.208169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e51583-13cb-4343-8d04-9c62bc240fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:22.208804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e51583-13cb-4343-8d04-9c62bc240fbb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:22.209082 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e51583-13cb-4343-8d04-9c62bc240fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:22.209343 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e51583-13cb-4343-8d04-9c62bc240fbb None admin] PATH_INFO: `/v3/users/glance` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:22.217922 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e51583-13cb-4343-8d04-9c62bc240fbb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:22.219315 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e51583-13cb-4343-8d04-9c62bc240fbb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:22.235368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d7e51583-13cb-4343-8d04-9c62bc240fbb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 08:42:22.235368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:22.235368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:22.235368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:22.235368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:22.235368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:22.235368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:22.235368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:22.235368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:22.235368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:22.235368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:22.235368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:22.235368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:22.235368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:22.235368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:22.235368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:22.236498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:22.236498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:22.236498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:22.236498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:22.236498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:22.236498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:22.236498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:22.236498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:22.236498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:22.236498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:22.236498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:22.236498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:22.236498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:22.236498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:22.236498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:22.236498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:22.236498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:22.236498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:22.237381 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:22.237381 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:22.237381 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:22.237381 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:22.237381 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:22.237381 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:22.237381 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:22.237381 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:22.237381 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 08:42:22.237381 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:22.237381 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 254/509] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:42:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:22.237927 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e51583-13cb-4343-8d04-9c62bc240fbb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:22.247792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512821a8-49c6-406d-9ccf-3dab0937c0d7 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:22.285977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd7a37-2560-49bc-a9ff-65f60008b309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:22.286943 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd7a37-2560-49bc-a9ff-65f60008b309 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:22.287230 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd7a37-2560-49bc-a9ff-65f60008b309 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:22.287504 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd7a37-2560-49bc-a9ff-65f60008b309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:22.302693 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd7a37-2560-49bc-a9ff-65f60008b309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:22.304511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd7a37-2560-49bc-a9ff-65f60008b309 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:22.326900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-4edd7a37-2560-49bc-a9ff-65f60008b309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:22.326900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:22.326900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:22.326900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:22.326900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:22.326900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:22.326900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:22.326900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:22.326900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:22.326900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:22.326900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:22.326900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:22.326900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:22.326900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:22.326900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:22.326900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:22.327904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:22.327904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:22.327904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:22.327904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:22.327904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:22.327904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:22.327904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:22.327904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:22.327904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:22.327904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:22.327904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:22.327904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:22.327904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:22.327904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:22.327904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:22.327904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:22.327904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:22.329452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:22.329452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:22.329452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:22.329452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 256/510] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:22.329452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd7a37-2560-49bc-a9ff-65f60008b309 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:22.351263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:22.369193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e51583-13cb-4343-8d04-9c62bc240fbb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:22.416571 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec325653-6df7-4481-ab7a-4b603e636401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:22.417205 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec325653-6df7-4481-ab7a-4b603e636401 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:22.417487 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec325653-6df7-4481-ab7a-4b603e636401 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:22.417767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec325653-6df7-4481-ab7a-4b603e636401 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:22.425994 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec325653-6df7-4481-ab7a-4b603e636401 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:22.427466 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec325653-6df7-4481-ab7a-4b603e636401 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:22.440004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ec325653-6df7-4481-ab7a-4b603e636401 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:42:22.440004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:22.440004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:22.440004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:22.440004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:22.440004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:22.440004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:22.440004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:22.440004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:22.440004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:22.440004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:22.440004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:22.440004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:22.440004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:22.440004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:22.440004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:22.441016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:22.441016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:22.441016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:22.441016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:22.441016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:22.441016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:22.441016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:22.441016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:22.441016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:22.441016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:22.441016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:22.441016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:22.441016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:22.441016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:22.441016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:22.441016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:22.441016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:22.441016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:22.442096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:22.442096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:22.442096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:22.442096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:22.442096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:22.442096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:22.442096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:22.442096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:22.442096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:42:22.442096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:22.442096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 255/511] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:42:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:22.442096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec325653-6df7-4481-ab7a-4b603e636401 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:22.454701 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd7a37-2560-49bc-a9ff-65f60008b309 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:22.456937 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:22.490120 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed58959-9eb9-4e29-9067-c765c3a6c5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:22.490644 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed58959-9eb9-4e29-9067-c765c3a6c5de None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:22.491436 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed58959-9eb9-4e29-9067-c765c3a6c5de None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:22.491708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed58959-9eb9-4e29-9067-c765c3a6c5de None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:22.492557 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed58959-9eb9-4e29-9067-c765c3a6c5de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:22.492864 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed58959-9eb9-4e29-9067-c765c3a6c5de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:22.494249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed58959-9eb9-4e29-9067-c765c3a6c5de None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:22.507211 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 257/512] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:22.508176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed58959-9eb9-4e29-9067-c765c3a6c5de None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:22.552808 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec325653-6df7-4481-ab7a-4b603e636401 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:22.588687 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:22.591843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb907dba-eca4-4260-a9e1-c04994ad3eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:22.592499 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb907dba-eca4-4260-a9e1-c04994ad3eaa None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:22.592777 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb907dba-eca4-4260-a9e1-c04994ad3eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:22.593072 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb907dba-eca4-4260-a9e1-c04994ad3eaa None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:22.593813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb907dba-eca4-4260-a9e1-c04994ad3eaa None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:22.594115 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb907dba-eca4-4260-a9e1-c04994ad3eaa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:22.595548 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb907dba-eca4-4260-a9e1-c04994ad3eaa None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:22.617389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed58959-9eb9-4e29-9067-c765c3a6c5de None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:22.634288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 256/513] 10.222.0.67 () {56 vars in 1137 bytes} [Wed Aug 24 08:42:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:22.635140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb907dba-eca4-4260-a9e1-c04994ad3eaa None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:22.658219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddcf65-f020-477f-8857-74d31499f81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:22.658827 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddcf65-f020-477f-8857-74d31499f81c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:22.659103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddcf65-f020-477f-8857-74d31499f81c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:22.659370 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddcf65-f020-477f-8857-74d31499f81c None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:22.660017 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daddcf65-f020-477f-8857-74d31499f81c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:22.660354 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daddcf65-f020-477f-8857-74d31499f81c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:22.661727 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daddcf65-f020-477f-8857-74d31499f81c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:22.674053 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 258/514] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:22.674966 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddcf65-f020-477f-8857-74d31499f81c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:22.706632 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:22.742795 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb907dba-eca4-4260-a9e1-c04994ad3eaa None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:22.783795 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3d733-9628-40cb-b4f7-436197a93682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:22.784846 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3d733-9628-40cb-b4f7-436197a93682 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:22.785328 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3d733-9628-40cb-b4f7-436197a93682 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:22.785790 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3d733-9628-40cb-b4f7-436197a93682 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:22.790045 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddcf65-f020-477f-8857-74d31499f81c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:22.797125 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3d733-9628-40cb-b4f7-436197a93682 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:22.798596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3d733-9628-40cb-b4f7-436197a93682 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:22.813560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-12e3d733-9628-40cb-b4f7-436197a93682 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:22.813560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:22.813560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:22.813560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:22.813560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:22.813560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:22.813560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:22.813560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:22.813560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:22.813560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:22.813560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:22.813560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:22.813560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:22.813560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:22.813560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:22.813560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:22.814686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:22.814686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:22.814686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:22.814686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:22.814686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:22.814686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:22.814686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:22.814686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:22.814686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:22.814686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:22.814686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:22.814686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:22.814686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:22.814686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:22.814686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:22.814686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:22.814686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:22.815527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:22.815527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:22.815527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:22.815527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 257/515] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:22.815527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3d733-9628-40cb-b4f7-436197a93682 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:22.830627 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f1479-8da2-437f-8502-ea452dd76b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:22.831523 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f1479-8da2-437f-8502-ea452dd76b26 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:22.831951 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f1479-8da2-437f-8502-ea452dd76b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:22.832263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f1479-8da2-437f-8502-ea452dd76b26 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:22.833002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f1479-8da2-437f-8502-ea452dd76b26 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:22.833301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f1479-8da2-437f-8502-ea452dd76b26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:22.835335 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f1479-8da2-437f-8502-ea452dd76b26 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:22.837668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:22.856320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 259/516] 10.222.0.67 () {56 vars in 1137 bytes} [Wed Aug 24 08:42:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:22.857158 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f1479-8da2-437f-8502-ea452dd76b26 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:22.924112 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3d733-9628-40cb-b4f7-436197a93682 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:22.945305 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:22.960605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc0958-25ee-4e63-b422-b4ccfd170c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:22.961243 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc0958-25ee-4e63-b422-b4ccfd170c82 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:22.961611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc0958-25ee-4e63-b422-b4ccfd170c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:22.961885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc0958-25ee-4e63-b422-b4ccfd170c82 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:22.964088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f1479-8da2-437f-8502-ea452dd76b26 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:22.970552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc0958-25ee-4e63-b422-b4ccfd170c82 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:22.972014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc0958-25ee-4e63-b422-b4ccfd170c82 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:22.983976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-c7bc0958-25ee-4e63-b422-b4ccfd170c82 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 08:42:22.983976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:22.983976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:22.983976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:22.983976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:22.983976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:22.983976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:22.983976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:22.983976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:22.983976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:22.983976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:22.983976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:22.983976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:22.983976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:22.983976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:22.983976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:22.985195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:22.985195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:22.985195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:22.985195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:22.985195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:22.985195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:22.985195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:22.985195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:22.985195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:22.985195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:22.985195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:22.985195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:22.985195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:22.985195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:22.985195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:22.985195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:22.985195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:22.985195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:22.986284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:22.986284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:22.986284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:22.986284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:22.986284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:22.986284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:22.986284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:22.986284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:22.986284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 08:42:22.986284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:22.986284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 258/517] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:22.986967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc0958-25ee-4e63-b422-b4ccfd170c82 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:23.002998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ebbfa-a2ee-4e8e-8486-0fd62345c788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:23.003541 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ebbfa-a2ee-4e8e-8486-0fd62345c788 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:23.003786 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ebbfa-a2ee-4e8e-8486-0fd62345c788 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:23.004055 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ebbfa-a2ee-4e8e-8486-0fd62345c788 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:23.066167 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:23.077072 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ebbfa-a2ee-4e8e-8486-0fd62345c788 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:23.078564 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ebbfa-a2ee-4e8e-8486-0fd62345c788 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:23.091691 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc0958-25ee-4e63-b422-b4ccfd170c82 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:23.132733 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f031067-4dbd-4b26-a4be-e32865000de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:23.133385 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f031067-4dbd-4b26-a4be-e32865000de2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:23.133666 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f031067-4dbd-4b26-a4be-e32865000de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:23.133938 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f031067-4dbd-4b26-a4be-e32865000de2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:23.141448 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f031067-4dbd-4b26-a4be-e32865000de2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:23.142934 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f031067-4dbd-4b26-a4be-e32865000de2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:23.156229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-6f031067-4dbd-4b26-a4be-e32865000de2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:23.156229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:23.156229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:23.156229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:23.156229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:23.156229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:23.156229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:23.156229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:23.156229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:23.156229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:23.156229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:23.156229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:23.156229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:23.156229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:23.156229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:23.162705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:23.162705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:23.162705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:23.162705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:23.162705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:23.162705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:23.162705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:23.162705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:23.162705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:23.162705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:23.162705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:23.162705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:23.162705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:23.162705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:23.162705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:23.162705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:23.162705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:23.164076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:23.164076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:23.164076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:23.164076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:23.164076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:23.164076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:23.164076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:23.164076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:23.164076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 259/518] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:23.164076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f031067-4dbd-4b26-a4be-e32865000de2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:23.191377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 260/519] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:42:23.192321 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ebbfa-a2ee-4e8e-8486-0fd62345c788 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:23.211766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:23.276293 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f031067-4dbd-4b26-a4be-e32865000de2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:23.297010 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:23.308897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ebbfa-a2ee-4e8e-8486-0fd62345c788 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:23.313993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9ea45-7529-4733-b983-dd36d9e1ffd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:23.314606 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9ea45-7529-4733-b983-dd36d9e1ffd4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:23.314881 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9ea45-7529-4733-b983-dd36d9e1ffd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:23.315150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9ea45-7529-4733-b983-dd36d9e1ffd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:23.315794 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a9ea45-7529-4733-b983-dd36d9e1ffd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:23.316102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a9ea45-7529-4733-b983-dd36d9e1ffd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:23.317487 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a9ea45-7529-4733-b983-dd36d9e1ffd4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:23.330835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 260/520] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:23.331540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9ea45-7529-4733-b983-dd36d9e1ffd4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:23.352510 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e758e0b-7131-4e1a-95e6-6d4eb74b2c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:23.353130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e758e0b-7131-4e1a-95e6-6d4eb74b2c63 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:23.353410 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e758e0b-7131-4e1a-95e6-6d4eb74b2c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:23.353928 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e758e0b-7131-4e1a-95e6-6d4eb74b2c63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:23.424691 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e758e0b-7131-4e1a-95e6-6d4eb74b2c63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:23.426390 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e758e0b-7131-4e1a-95e6-6d4eb74b2c63 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:23.437322 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:23.440623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9ea45-7529-4733-b983-dd36d9e1ffd4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:23.478818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0051cf5a-91b3-425f-91b6-68bcfd4e1e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:23.479436 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051cf5a-91b3-425f-91b6-68bcfd4e1e58 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:23.479715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051cf5a-91b3-425f-91b6-68bcfd4e1e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:23.479978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051cf5a-91b3-425f-91b6-68bcfd4e1e58 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:23.480745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0051cf5a-91b3-425f-91b6-68bcfd4e1e58 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:23.481048 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0051cf5a-91b3-425f-91b6-68bcfd4e1e58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:23.482504 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0051cf5a-91b3-425f-91b6-68bcfd4e1e58 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:23.502720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 261/521] 10.222.0.67 () {56 vars in 1139 bytes} [Wed Aug 24 08:42:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:23.503579 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0051cf5a-91b3-425f-91b6-68bcfd4e1e58 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:23.530125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:23.541576 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 261/522] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:42:23.542221 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e758e0b-7131-4e1a-95e6-6d4eb74b2c63 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:23.625711 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0051cf5a-91b3-425f-91b6-68bcfd4e1e58 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:23.656753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e758e0b-7131-4e1a-95e6-6d4eb74b2c63 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:23.676806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf34e23-a38e-46c9-9676-728e417b58ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:23.677491 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf34e23-a38e-46c9-9676-728e417b58ce None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:23.677745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf34e23-a38e-46c9-9676-728e417b58ce None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:23.677986 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf34e23-a38e-46c9-9676-728e417b58ce None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:23.678659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf34e23-a38e-46c9-9676-728e417b58ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:23.679470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf34e23-a38e-46c9-9676-728e417b58ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:23.680953 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf34e23-a38e-46c9-9676-728e417b58ce None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:23.700276 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 262/523] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:23.703717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf34e23-a38e-46c9-9676-728e417b58ce None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:23.706942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa13b17-30aa-480d-b30b-7916045fc8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:23.707593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa13b17-30aa-480d-b30b-7916045fc8de None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:23.707871 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa13b17-30aa-480d-b30b-7916045fc8de None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:23.708144 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa13b17-30aa-480d-b30b-7916045fc8de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:23.722636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa13b17-30aa-480d-b30b-7916045fc8de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:23.724349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa13b17-30aa-480d-b30b-7916045fc8de None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:23.726598 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:23.745347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-0fa13b17-30aa-480d-b30b-7916045fc8de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:23.745347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:23.745347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:23.745347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:23.745347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:23.745347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:23.745347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:23.745347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:23.745347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:23.745347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:23.745347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:23.745347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:23.745347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:23.745347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:23.745347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:23.745347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:23.753197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:23.753197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:23.753197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:23.753197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:23.753197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:23.753197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:23.753197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:23.753197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:23.753197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:23.753197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:23.753197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:23.753197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:23.753197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:23.753197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:23.753197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:23.753197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:23.753197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:23.754258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:23.754258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:23.754258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:23.754258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 262/524] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:23.754258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa13b17-30aa-480d-b30b-7916045fc8de None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:23.804733 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:23.823023 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf34e23-a38e-46c9-9676-728e417b58ce None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:23.861887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfc76b-bf6b-4367-9c6d-e6cacbe0085f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:23.862499 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfc76b-bf6b-4367-9c6d-e6cacbe0085f None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:23.862776 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfc76b-bf6b-4367-9c6d-e6cacbe0085f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:23.863053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfc76b-bf6b-4367-9c6d-e6cacbe0085f None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:23.863688 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfc76b-bf6b-4367-9c6d-e6cacbe0085f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:23.865164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfc76b-bf6b-4367-9c6d-e6cacbe0085f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:23.881464 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa13b17-30aa-480d-b30b-7916045fc8de None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:23.887616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 263/525] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 24 08:42:23 2022] POST /identity/v3/users => generated 247 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:42:23.888485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfc76b-bf6b-4367-9c6d-e6cacbe0085f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:23.941225 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8565c-8eb2-4b92-9311-c146c3c75e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:23.942558 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8565c-8eb2-4b92-9311-c146c3c75e6c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:23.942558 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8565c-8eb2-4b92-9311-c146c3c75e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:23.942558 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8565c-8eb2-4b92-9311-c146c3c75e6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:23.992312 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:24.031481 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfc76b-bf6b-4367-9c6d-e6cacbe0085f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:24.035537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8565c-8eb2-4b92-9311-c146c3c75e6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:24.037180 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8565c-8eb2-4b92-9311-c146c3c75e6c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:24.081166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc58f4-2da5-46c6-90bc-79cb86ddc07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:24.081731 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc58f4-2da5-46c6-90bc-79cb86ddc07d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:24.082066 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc58f4-2da5-46c6-90bc-79cb86ddc07d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:24.082877 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc58f4-2da5-46c6-90bc-79cb86ddc07d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:24.090652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:24.098329 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc58f4-2da5-46c6-90bc-79cb86ddc07d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:24.099857 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc58f4-2da5-46c6-90bc-79cb86ddc07d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:24.115696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-08cc58f4-2da5-46c6-90bc-79cb86ddc07d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:24.115696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:24.115696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:24.115696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:24.115696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:24.115696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:24.115696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:24.115696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:24.115696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:24.115696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:24.115696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:24.115696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:24.115696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:24.115696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:24.115696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:24.115696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:24.117314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:24.117314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:24.117314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:24.117314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:24.117314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:24.117314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:24.117314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:24.117314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:24.117314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:24.117314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:24.117314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:24.117314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:24.117314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:24.117314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:24.117314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:24.117314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:24.117314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:24.118611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:24.118611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:24.118611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:24.118611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 264/526] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:24.118611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc58f4-2da5-46c6-90bc-79cb86ddc07d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:24.168788 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 263/527] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:42:24.169676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8565c-8eb2-4b92-9311-c146c3c75e6c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:24.255280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc58f4-2da5-46c6-90bc-79cb86ddc07d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:24.285951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:24.295792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07488a94-5df1-4bef-80b9-66e241d8bbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:24.296437 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07488a94-5df1-4bef-80b9-66e241d8bbb8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:24.296740 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07488a94-5df1-4bef-80b9-66e241d8bbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:24.296986 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07488a94-5df1-4bef-80b9-66e241d8bbb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:24.297729 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07488a94-5df1-4bef-80b9-66e241d8bbb8 None admin] RBAC: Adding query filter params (role.id=9aa748c8ea2542e6987e0b93190ea4e6, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=ca56dd7216fb4946946b2801e144fec2) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:24.298038 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07488a94-5df1-4bef-80b9-66e241d8bbb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:24.299570 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07488a94-5df1-4bef-80b9-66e241d8bbb8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:24.316125 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 265/528] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:42:24 2022] GET /identity/v3/role_assignments?user.id=ca56dd7216fb4946946b2801e144fec2&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=9aa748c8ea2542e6987e0b93190ea4e6 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:24.316125 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07488a94-5df1-4bef-80b9-66e241d8bbb8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:24.340951 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8565c-8eb2-4b92-9311-c146c3c75e6c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:24.412088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951fd09-2846-4692-ad7a-027891247578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:24.414330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951fd09-2846-4692-ad7a-027891247578 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:24.414997 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951fd09-2846-4692-ad7a-027891247578 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:24.415555 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951fd09-2846-4692-ad7a-027891247578 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:24.416885 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9951fd09-2846-4692-ad7a-027891247578 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:24.417510 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9951fd09-2846-4692-ad7a-027891247578 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:24.421748 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9951fd09-2846-4692-ad7a-027891247578 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:24.449506 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 264/529] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:24.449506 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951fd09-2846-4692-ad7a-027891247578 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:24.455610 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:24.477407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07488a94-5df1-4bef-80b9-66e241d8bbb8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:24.522253 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78a738-61c2-44de-9318-72fedfacf973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:24.523147 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78a738-61c2-44de-9318-72fedfacf973 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:24.523578 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78a738-61c2-44de-9318-72fedfacf973 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:24.524030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78a738-61c2-44de-9318-72fedfacf973 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:24.524898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78a738-61c2-44de-9318-72fedfacf973 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:24.525359 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78a738-61c2-44de-9318-72fedfacf973 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:24.527037 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78a738-61c2-44de-9318-72fedfacf973 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:24.543439 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 266/530] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:24.544774 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78a738-61c2-44de-9318-72fedfacf973 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:24.620255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:24.642651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951fd09-2846-4692-ad7a-027891247578 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:24.696021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78a738-61c2-44de-9318-72fedfacf973 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:24.707619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb0fd3-8fc0-4f0c-bafc-0a899558fb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:24.709195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb0fd3-8fc0-4f0c-bafc-0a899558fb68 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:24.710219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb0fd3-8fc0-4f0c-bafc-0a899558fb68 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:24.711294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb0fd3-8fc0-4f0c-bafc-0a899558fb68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:24.727785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb0fd3-8fc0-4f0c-bafc-0a899558fb68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:24.730329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb0fd3-8fc0-4f0c-bafc-0a899558fb68 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:24.742377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747d16e-d21f-4e1e-90e9-c73826d1e610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:24.743039 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747d16e-d21f-4e1e-90e9-c73826d1e610 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:24.743321 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747d16e-d21f-4e1e-90e9-c73826d1e610 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:24.743594 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747d16e-d21f-4e1e-90e9-c73826d1e610 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:24.751778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747d16e-d21f-4e1e-90e9-c73826d1e610 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:24.752601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-60fb0fd3-8fc0-4f0c-bafc-0a899558fb68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:24.752601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:24.752601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:24.752601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:24.752601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:24.752601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:24.752601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:24.752601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:24.752601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:24.752601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:24.752601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:24.752601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:24.752601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:24.752601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:24.752601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:24.752601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:24.753483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:24.753483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:24.753483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:24.753483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:24.753483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:24.753483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:24.753483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:24.753483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:24.753483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:24.753483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:24.753483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:24.753483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:24.753483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:24.753483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:24.753483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:24.753483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:24.753483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:24.754329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:24.754329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:24.754329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:24.754488 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747d16e-d21f-4e1e-90e9-c73826d1e610 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:24.756316 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 265/531] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:24.761330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb0fd3-8fc0-4f0c-bafc-0a899558fb68 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:24.773650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-9747d16e-d21f-4e1e-90e9-c73826d1e610 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:24.773650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:24.773650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:24.773650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:24.773650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:24.773650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:24.773650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:24.773650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:24.773650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:24.773650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:24.773650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:24.773650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:24.773650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:24.773650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:24.773650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:24.774593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:24.774593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:24.774593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:24.774593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:24.774593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:24.774593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:24.774593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:24.774593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:24.774593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:24.774593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:24.774593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:24.774593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:24.774593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:24.774593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:24.774593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:24.774593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:24.774593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:24.775417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:24.775417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:24.775417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:24.775417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:24.775417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:24.775417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:24.775417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:24.775417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:24.779889 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 267/532] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:24.780886 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747d16e-d21f-4e1e-90e9-c73826d1e610 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:24.847557 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:24.902192 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb0fd3-8fc0-4f0c-bafc-0a899558fb68 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:24.912506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747d16e-d21f-4e1e-90e9-c73826d1e610 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:24.927386 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:24.947581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290eb29-d016-458b-81cb-750d33a03468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:24.948401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290eb29-d016-458b-81cb-750d33a03468 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:24.948689 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290eb29-d016-458b-81cb-750d33a03468 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:24.948966 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290eb29-d016-458b-81cb-750d33a03468 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:24.959594 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290eb29-d016-458b-81cb-750d33a03468 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:24.962312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290eb29-d016-458b-81cb-750d33a03468 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:24.962905 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3532595-ac1f-4391-a8e0-935cd998ef57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:24.963801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3532595-ac1f-4391-a8e0-935cd998ef57 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:24.964287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3532595-ac1f-4391-a8e0-935cd998ef57 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:24.965037 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3532595-ac1f-4391-a8e0-935cd998ef57 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:24.966338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3532595-ac1f-4391-a8e0-935cd998ef57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:24.966833 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3532595-ac1f-4391-a8e0-935cd998ef57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:24.968428 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3532595-ac1f-4391-a8e0-935cd998ef57 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:24.977101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-2290eb29-d016-458b-81cb-750d33a03468 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:24.977101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:24.977101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:24.977101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:24.977101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:24.977101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:24.977101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:24.977101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:24.977101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:24.977101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:24.977101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:24.977101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:24.977101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:24.977101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:24.977101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:24.978014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:24.978014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:24.978014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:24.978014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:24.978014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:24.978014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:24.978014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:24.978014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:24.978014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:24.978014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:24.978014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:24.978014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:24.978014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:24.978014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:24.978014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:24.978014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:24.978014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:24.978861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:24.978861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:24.978861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:24.978861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:24.978861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:24.978861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:24.978861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:24.978861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:24.978861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 266/533] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:24.979996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290eb29-d016-458b-81cb-750d33a03468 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:24.988174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 268/534] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:24.993676 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3532595-ac1f-4391-a8e0-935cd998ef57 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:25.105015 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290eb29-d016-458b-81cb-750d33a03468 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:25.127681 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:25.138005 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3532595-ac1f-4391-a8e0-935cd998ef57 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:25.155317 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcf0ba-9086-44fa-bb1a-7ca752aaf362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:25.156513 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcf0ba-9086-44fa-bb1a-7ca752aaf362 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:25.156968 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcf0ba-9086-44fa-bb1a-7ca752aaf362 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:25.157434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcf0ba-9086-44fa-bb1a-7ca752aaf362 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:25.158560 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcf0ba-9086-44fa-bb1a-7ca752aaf362 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:25.159446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcf0ba-9086-44fa-bb1a-7ca752aaf362 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:25.161083 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcf0ba-9086-44fa-bb1a-7ca752aaf362 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:25.182807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbac59-b347-4d67-8d20-e608b6125b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:25.183972 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbac59-b347-4d67-8d20-e608b6125b4f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:25.184665 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbac59-b347-4d67-8d20-e608b6125b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:25.185105 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbac59-b347-4d67-8d20-e608b6125b4f None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:25.186066 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbac59-b347-4d67-8d20-e608b6125b4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:25.186591 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbac59-b347-4d67-8d20-e608b6125b4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:25.188355 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbac59-b347-4d67-8d20-e608b6125b4f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:25.190058 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 267/535] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:25.191050 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcf0ba-9086-44fa-bb1a-7ca752aaf362 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:25.204258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 269/536] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:25.205314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbac59-b347-4d67-8d20-e608b6125b4f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:25.241057 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:25.325483 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbac59-b347-4d67-8d20-e608b6125b4f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:25.330252 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcf0ba-9086-44fa-bb1a-7ca752aaf362 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:25.368712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474cabf-a324-4926-9089-f24b9c05f234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:25.370219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474cabf-a324-4926-9089-f24b9c05f234 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:25.370796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474cabf-a324-4926-9089-f24b9c05f234 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:25.371598 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474cabf-a324-4926-9089-f24b9c05f234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:25.372990 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474cabf-a324-4926-9089-f24b9c05f234 None admin] RBAC: Adding query filter params (role.id=9aa748c8ea2542e6987e0b93190ea4e6, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=5979387f7aa54619b8d429368c306cc9) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:25.373648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474cabf-a324-4926-9089-f24b9c05f234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:25.376636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474cabf-a324-4926-9089-f24b9c05f234 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:25.377317 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66794e7c-13e2-4c07-b0f5-da0e01c32410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:25.378174 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66794e7c-13e2-4c07-b0f5-da0e01c32410 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:25.378624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66794e7c-13e2-4c07-b0f5-da0e01c32410 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:25.379064 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66794e7c-13e2-4c07-b0f5-da0e01c32410 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:25.388585 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 270/537] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:42:25 2022] GET /identity/v3/role_assignments?user.id=5979387f7aa54619b8d429368c306cc9&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=9aa748c8ea2542e6987e0b93190ea4e6 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:25.390115 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66794e7c-13e2-4c07-b0f5-da0e01c32410 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:25.391966 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66794e7c-13e2-4c07-b0f5-da0e01c32410 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:25.399746 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474cabf-a324-4926-9089-f24b9c05f234 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:25.416998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-66794e7c-13e2-4c07-b0f5-da0e01c32410 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:25.416998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:25.416998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:25.416998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:25.416998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:25.416998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:25.416998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:25.416998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:25.416998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:25.416998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:25.416998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:25.416998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:25.416998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:25.416998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:25.416998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:25.417874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:25.417874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:25.417874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:25.417874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:25.417874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:25.417874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:25.417874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:25.417874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:25.417874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:25.417874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:25.417874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:25.417874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:25.417874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:25.417874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:25.417874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:25.417874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:25.417874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:25.418700 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:25.418700 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:25.418700 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:25.418700 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:25.418700 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:25.418700 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:25.418700 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:25.418700 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:25.421896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 268/538] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:25.429440 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66794e7c-13e2-4c07-b0f5-da0e01c32410 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:25.480083 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:25.573025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66794e7c-13e2-4c07-b0f5-da0e01c32410 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:25.620153 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931f04a-11c1-409e-a15b-92fcbe98db89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:25.621229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931f04a-11c1-409e-a15b-92fcbe98db89 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:25.621726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931f04a-11c1-409e-a15b-92fcbe98db89 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:25.622241 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931f04a-11c1-409e-a15b-92fcbe98db89 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:25.623270 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931f04a-11c1-409e-a15b-92fcbe98db89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:25.623832 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931f04a-11c1-409e-a15b-92fcbe98db89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:25.625627 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931f04a-11c1-409e-a15b-92fcbe98db89 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:25.644901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 269/539] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:25.653043 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931f04a-11c1-409e-a15b-92fcbe98db89 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:25.664952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:25.682646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474cabf-a324-4926-9089-f24b9c05f234 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:25.734471 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a23953-9734-4557-a110-46d7cd31b5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:25.735104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a23953-9734-4557-a110-46d7cd31b5a2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:25.735382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a23953-9734-4557-a110-46d7cd31b5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:25.735652 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a23953-9734-4557-a110-46d7cd31b5a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:25.736477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a23953-9734-4557-a110-46d7cd31b5a2 None admin] RBAC: Adding query filter params (role.id=9aa748c8ea2542e6987e0b93190ea4e6, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=5de1586efc8c4dec898b0fb54b56acf0) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:25.736842 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a23953-9734-4557-a110-46d7cd31b5a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:25.738497 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a23953-9734-4557-a110-46d7cd31b5a2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:25.746207 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 271/540] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:42:25 2022] GET /identity/v3/role_assignments?user.id=5de1586efc8c4dec898b0fb54b56acf0&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=9aa748c8ea2542e6987e0b93190ea4e6 => generated 259 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:25.803629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:25.814310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931f04a-11c1-409e-a15b-92fcbe98db89 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:25.877398 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c49a0-5607-486b-8b02-f39d4ebef5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:25.878027 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c49a0-5607-486b-8b02-f39d4ebef5de None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:25.878307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c49a0-5607-486b-8b02-f39d4ebef5de None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:25.878574 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c49a0-5607-486b-8b02-f39d4ebef5de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:25.879327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c49a0-5607-486b-8b02-f39d4ebef5de None admin] RBAC: Adding query filter params (role.id=9aa748c8ea2542e6987e0b93190ea4e6, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=2ee2e93901914b1d82607a3ae0e2acf1) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:25.879637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c49a0-5607-486b-8b02-f39d4ebef5de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:25.881170 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c49a0-5607-486b-8b02-f39d4ebef5de None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:25.888682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 270/541] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:42:25 2022] GET /identity/v3/role_assignments?user.id=2ee2e93901914b1d82607a3ae0e2acf1&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=9aa748c8ea2542e6987e0b93190ea4e6 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:25.998271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f9d7f-3745-4546-81a3-a6514bda4d4c None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:25.998451 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f9d7f-3745-4546-81a3-a6514bda4d4c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:25.998720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f9d7f-3745-4546-81a3-a6514bda4d4c None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:26.000514 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 272/542] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:26.007184 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635be8bc-5903-4062-919d-1d892b6ff636 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:26.007841 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635be8bc-5903-4062-919d-1d892b6ff636 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:26.008363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635be8bc-5903-4062-919d-1d892b6ff636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:26.029783 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-635be8bc-5903-4062-919d-1d892b6ff636 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:26.031479 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-635be8bc-5903-4062-919d-1d892b6ff636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:26.128462 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 271/543] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:26.422961 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f9d7f-3745-4546-81a3-a6514bda4d4c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:26.522476 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316594a-edda-4bdd-9a26-3ec73dbf4ba0 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:26.522980 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316594a-edda-4bdd-9a26-3ec73dbf4ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:26.523346 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316594a-edda-4bdd-9a26-3ec73dbf4ba0 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:26.525014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 272/544] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:26.530822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1961f74-2838-4b97-994c-c33e52a498f7 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:26.531170 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1961f74-2838-4b97-994c-c33e52a498f7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:26.531389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1961f74-2838-4b97-994c-c33e52a498f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:26.568906 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f9d7f-3745-4546-81a3-a6514bda4d4c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:26.576791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-c1961f74-2838-4b97-994c-c33e52a498f7 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:26.577997 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c1961f74-2838-4b97-994c-c33e52a498f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:26.670772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f07df-bb2e-4dc5-b17e-0c361ce543f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:26.671770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f07df-bb2e-4dc5-b17e-0c361ce543f8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:26.672307 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f07df-bb2e-4dc5-b17e-0c361ce543f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:26.678599 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f07df-bb2e-4dc5-b17e-0c361ce543f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:26.712557 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f07df-bb2e-4dc5-b17e-0c361ce543f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:26.724843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-3e3f07df-bb2e-4dc5-b17e-0c361ce543f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:26.724843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:26.724843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:26.724843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:26.724843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:26.724843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:26.724843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:26.724843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:26.724843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:26.724843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:26.724843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:26.724843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:26.724843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:26.724843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:26.724843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:26.724843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:26.725854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:26.725854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:26.725854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:26.725854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:26.725854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:26.725854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:26.725854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:26.725854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:26.725854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:26.725854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:26.725854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:26.725854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:26.725854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:26.725854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:26.725854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:26.725854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:26.726651 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:26.726651 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:26.726651 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:26.726651 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:26.726651 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:26.726651 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:26.726651 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:26.726651 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:26.726651 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:26.726651 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:26.726651 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:26.726651 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:26.727254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 273/545] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:26.730607 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 273/546] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:42:26.740115 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f07df-bb2e-4dc5-b17e-0c361ce543f8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:26.785418 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:26.917271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f07df-bb2e-4dc5-b17e-0c361ce543f8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:26.971208 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f29c8-bf54-4d08-858d-098619140155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:26.971737 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f29c8-bf54-4d08-858d-098619140155 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:26.972013 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f29c8-bf54-4d08-858d-098619140155 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:26.972304 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f29c8-bf54-4d08-858d-098619140155 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:26.972976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f29c8-bf54-4d08-858d-098619140155 None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:26.973275 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f29c8-bf54-4d08-858d-098619140155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:26.974628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f29c8-bf54-4d08-858d-098619140155 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:26.987394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 274/547] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:42:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:26.993298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1961f74-2838-4b97-994c-c33e52a498f7 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:27.023581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:27.121188 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1961f74-2838-4b97-994c-c33e52a498f7 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:27.173442 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05107c-ddd2-404e-80bc-1e1e640922db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:27.174103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05107c-ddd2-404e-80bc-1e1e640922db None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:27.174350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05107c-ddd2-404e-80bc-1e1e640922db None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:27.174621 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05107c-ddd2-404e-80bc-1e1e640922db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:27.192226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05107c-ddd2-404e-80bc-1e1e640922db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:27.193643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05107c-ddd2-404e-80bc-1e1e640922db None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:27.214258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-fc05107c-ddd2-404e-80bc-1e1e640922db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:27.214258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:27.214258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:27.214258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:27.214258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:27.214258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:27.214258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:27.214258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:27.214258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:27.214258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:27.214258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:27.214258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:27.214258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:27.214258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:27.214258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:27.214258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:27.215327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:27.215327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:27.215327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:27.215327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:27.215327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:27.215327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:27.215327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:27.215327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:27.215327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:27.215327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:27.215327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:27.215327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:27.215327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:27.215327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:27.215327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:27.215327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:27.215327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:27.216299 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:27.216299 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:27.216299 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:27.216299 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 274/548] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:27.223884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f29c8-bf54-4d08-858d-098619140155 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:27.267184 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05107c-ddd2-404e-80bc-1e1e640922db None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:27.364707 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:27.404738 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f29c8-bf54-4d08-858d-098619140155 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:27.418013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:27.464872 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05107c-ddd2-404e-80bc-1e1e640922db None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:27.481711 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836114b5-5fa2-4cf6-ab0a-362f1ab2b412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:27.482326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836114b5-5fa2-4cf6-ab0a-362f1ab2b412 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:27.482560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836114b5-5fa2-4cf6-ab0a-362f1ab2b412 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:27.482791 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836114b5-5fa2-4cf6-ab0a-362f1ab2b412 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:27.483563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836114b5-5fa2-4cf6-ab0a-362f1ab2b412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:27.483888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836114b5-5fa2-4cf6-ab0a-362f1ab2b412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:27.485458 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836114b5-5fa2-4cf6-ab0a-362f1ab2b412 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:27.506613 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94732d-23ce-4367-9ff6-aaa1b193a04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:27.507378 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94732d-23ce-4367-9ff6-aaa1b193a04e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:27.507520 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94732d-23ce-4367-9ff6-aaa1b193a04e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:27.507793 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94732d-23ce-4367-9ff6-aaa1b193a04e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:27.513050 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 275/549] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:27.514594 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836114b5-5fa2-4cf6-ab0a-362f1ab2b412 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:27.520299 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94732d-23ce-4367-9ff6-aaa1b193a04e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:27.525838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-eb94732d-23ce-4367-9ff6-aaa1b193a04e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:27.525838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:27.525838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:27.525838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:27.525838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:27.525838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:27.525838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:27.525838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:27.525838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:27.525838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:27.525838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:27.525838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:27.525838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:27.525838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:27.525838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:27.525838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:27.527332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:27.527332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:27.527332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:27.527332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:27.527332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:27.527332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:27.527332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:27.527332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:27.527332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:27.527332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:27.527332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:27.527332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:27.527332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:27.527332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:27.527332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:27.527332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:27.528821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:27.528821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:27.528821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:27.528821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:27.528821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:27.528821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:27.528821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:27.528821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:27.528821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:27.528821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:27.528821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:27.528821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:27.528821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 275/550] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:42:27.535994 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94732d-23ce-4367-9ff6-aaa1b193a04e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:27.654437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94732d-23ce-4367-9ff6-aaa1b193a04e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:27.688052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:27.692708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da74c8-7559-4076-a468-a1f57d392332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:27.693380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da74c8-7559-4076-a468-a1f57d392332 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:27.693753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da74c8-7559-4076-a468-a1f57d392332 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:27.694121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da74c8-7559-4076-a468-a1f57d392332 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:27.694867 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da74c8-7559-4076-a468-a1f57d392332 None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:27.695296 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da74c8-7559-4076-a468-a1f57d392332 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:27.696746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da74c8-7559-4076-a468-a1f57d392332 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:27.707096 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 276/551] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:42:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:27.707315 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836114b5-5fa2-4cf6-ab0a-362f1ab2b412 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:27.721343 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da74c8-7559-4076-a468-a1f57d392332 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:27.765794 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13575981-7656-4405-8fb1-e29113958295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:27.766552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13575981-7656-4405-8fb1-e29113958295 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:27.766836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13575981-7656-4405-8fb1-e29113958295 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:27.767109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13575981-7656-4405-8fb1-e29113958295 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:27.785186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13575981-7656-4405-8fb1-e29113958295 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:27.786698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13575981-7656-4405-8fb1-e29113958295 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:27.790468 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:27.813479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-13575981-7656-4405-8fb1-e29113958295 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:42:27.813479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:27.813479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:27.813479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:27.813479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:27.813479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:27.813479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:27.813479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:27.813479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:27.813479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:27.813479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:27.813479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:27.813479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:27.813479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:27.813479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:27.813479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:27.815798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:27.815798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:27.815798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:27.815798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:27.815798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:27.815798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:27.815798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:27.815798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:27.815798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:27.815798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:27.815798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:27.815798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:27.815798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:27.815798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:27.815798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:27.815798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:27.815798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:27.815798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:27.821560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:27.821560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:27.821560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:27.821560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:27.821560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:27.821560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:27.821560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:27.821560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:27.821560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:42:27.821560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:27.821560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 276/552] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:42:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:27.822993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13575981-7656-4405-8fb1-e29113958295 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:27.883884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da74c8-7559-4076-a468-a1f57d392332 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:27.936047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d698b-dc7c-43f8-9c59-c10b2d426dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:27.936462 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d698b-dc7c-43f8-9c59-c10b2d426dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:27.936745 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d698b-dc7c-43f8-9c59-c10b2d426dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:27.937017 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d698b-dc7c-43f8-9c59-c10b2d426dd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:27.950327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d698b-dc7c-43f8-9c59-c10b2d426dd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:27.952062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d698b-dc7c-43f8-9c59-c10b2d426dd7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:27.972135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-fd5d698b-dc7c-43f8-9c59-c10b2d426dd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:27.972135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:27.972135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:27.972135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:27.972135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:27.972135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:27.972135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:27.972135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:27.972135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:27.972135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:27.972135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:27.972135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:27.972135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:27.972135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:27.972135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:27.972135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:27.973155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:27.973155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:27.973155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:27.973155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:27.973155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:27.973155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:27.973155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:27.973155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:27.973155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:27.973155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:27.973155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:27.973155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:27.973155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:27.973155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:27.973155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:27.973155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:27.973155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:27.974098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:27.974098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:27.974098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:27.974098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 277/553] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:27.980652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1fe10-041c-479b-a183-746cdf37aea6 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:27.980652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1fe10-041c-479b-a183-746cdf37aea6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:27.980652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1fe10-041c-479b-a183-746cdf37aea6 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:27.980652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 278/554] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:27.980652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1fe10-041c-479b-a183-746cdf37aea6 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:27.993196 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13575981-7656-4405-8fb1-e29113958295 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:28.044184 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faed548-6faa-462a-9766-3f05a72632cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:28.045183 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faed548-6faa-462a-9766-3f05a72632cb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:28.045670 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faed548-6faa-462a-9766-3f05a72632cb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:28.046619 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faed548-6faa-462a-9766-3f05a72632cb None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:28.048021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faed548-6faa-462a-9766-3f05a72632cb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:28.048567 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faed548-6faa-462a-9766-3f05a72632cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:28.051027 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faed548-6faa-462a-9766-3f05a72632cb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:28.052388 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:28.084377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 277/555] 10.222.0.67 () {56 vars in 1135 bytes} [Wed Aug 24 08:42:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:28.086042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb09da8-8da9-4b82-990e-3f42a167ad14 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:28.086302 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb09da8-8da9-4b82-990e-3f42a167ad14 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:28.086576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb09da8-8da9-4b82-990e-3f42a167ad14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:28.108291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-feb09da8-8da9-4b82-990e-3f42a167ad14 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:28.109541 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-feb09da8-8da9-4b82-990e-3f42a167ad14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:28.141900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1fe10-041c-479b-a183-746cdf37aea6 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:28.182716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:28.201953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5468e3c-1fd1-4be4-b287-9f948d02e14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:28.202747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5468e3c-1fd1-4be4-b287-9f948d02e14c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:28.203232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5468e3c-1fd1-4be4-b287-9f948d02e14c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:28.203836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5468e3c-1fd1-4be4-b287-9f948d02e14c None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:28.205123 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5468e3c-1fd1-4be4-b287-9f948d02e14c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:28.205547 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5468e3c-1fd1-4be4-b287-9f948d02e14c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:28.207453 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5468e3c-1fd1-4be4-b287-9f948d02e14c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:28.211132 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 278/556] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:28.212490 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb09da8-8da9-4b82-990e-3f42a167ad14 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:28.248766 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 279/557] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:28.253605 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5468e3c-1fd1-4be4-b287-9f948d02e14c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:28.366109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb09da8-8da9-4b82-990e-3f42a167ad14 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:28.398864 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5468e3c-1fd1-4be4-b287-9f948d02e14c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:28.405610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c996afd-b4e4-43a2-b94e-13c240f58870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:28.406586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c996afd-b4e4-43a2-b94e-13c240f58870 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:28.407092 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c996afd-b4e4-43a2-b94e-13c240f58870 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:28.407671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c996afd-b4e4-43a2-b94e-13c240f58870 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:28.408191 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:28.446502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be0923-4854-4644-96a6-05e207db026f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:28.447282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be0923-4854-4644-96a6-05e207db026f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:28.447681 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be0923-4854-4644-96a6-05e207db026f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:28.447976 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be0923-4854-4644-96a6-05e207db026f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:28.456965 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be0923-4854-4644-96a6-05e207db026f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:28.459062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be0923-4854-4644-96a6-05e207db026f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:28.476357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-49be0923-4854-4644-96a6-05e207db026f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:42:28.476357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:28.476357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:28.476357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:28.476357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:28.476357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:28.476357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:28.476357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:28.476357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:28.476357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:28.476357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:28.476357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:28.476357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:28.476357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:28.476357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:28.476357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:28.477991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:28.477991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:28.477991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:28.477991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:28.477991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:28.477991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:28.477991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:28.477991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:28.477991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:28.477991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:28.477991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:28.477991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:28.477991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:28.477991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:28.477991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:28.477991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:28.477991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:28.477991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:28.478962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:28.478962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:28.478962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:28.478962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:28.478962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:28.478962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:28.478962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:28.478962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:28.478962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:42:28.478962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:28.478962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 280/558] 10.222.0.67 () {56 vars in 1093 bytes} [Wed Aug 24 08:42:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:42:28.479571 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60233885-76cf-4bca-9e6f-0ad4246be5c9 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:28.479571 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60233885-76cf-4bca-9e6f-0ad4246be5c9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:28.479710 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60233885-76cf-4bca-9e6f-0ad4246be5c9 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:28.484181 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 281/559] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:28.485589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60233885-76cf-4bca-9e6f-0ad4246be5c9 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:28.487738 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:28.517963 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c996afd-b4e4-43a2-b94e-13c240f58870 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:28.520349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c996afd-b4e4-43a2-b94e-13c240f58870 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:28.619054 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60233885-76cf-4bca-9e6f-0ad4246be5c9 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:28.640514 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 279/560] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:42:28.641848 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f5e3c-9ff8-4ac0-b4f6-c60e0b3b9164 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:28.642705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f5e3c-9ff8-4ac0-b4f6-c60e0b3b9164 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:28.642705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f5e3c-9ff8-4ac0-b4f6-c60e0b3b9164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:28.657993 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b69d38-d5fc-41db-b18d-654eee69638a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:28.658604 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b69d38-d5fc-41db-b18d-654eee69638a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:28.658877 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b69d38-d5fc-41db-b18d-654eee69638a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:28.659143 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b69d38-d5fc-41db-b18d-654eee69638a None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:28.659869 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b69d38-d5fc-41db-b18d-654eee69638a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:28.660197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b69d38-d5fc-41db-b18d-654eee69638a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:28.661636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b69d38-d5fc-41db-b18d-654eee69638a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:28.664873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-531f5e3c-9ff8-4ac0-b4f6-c60e0b3b9164 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:28.666094 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-531f5e3c-9ff8-4ac0-b4f6-c60e0b3b9164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:28.681515 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 282/561] 10.222.0.67 () {56 vars in 1133 bytes} [Wed Aug 24 08:42:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:28.682374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b69d38-d5fc-41db-b18d-654eee69638a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:28.711342 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:28.760126 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 280/562] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:28.761021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb160a-113e-4c7a-b158-90600ea7082f None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:28.761305 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb160a-113e-4c7a-b158-90600ea7082f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:28.761576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb160a-113e-4c7a-b158-90600ea7082f None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:28.763031 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 281/563] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:28.763549 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb160a-113e-4c7a-b158-90600ea7082f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:28.791797 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:28.808615 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b69d38-d5fc-41db-b18d-654eee69638a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:28.852260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16921f9e-d891-4447-ab31-3bd8dee82bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:28.852998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16921f9e-d891-4447-ab31-3bd8dee82bbe None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:28.853340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16921f9e-d891-4447-ab31-3bd8dee82bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:28.853676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16921f9e-d891-4447-ab31-3bd8dee82bbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:28.866166 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16921f9e-d891-4447-ab31-3bd8dee82bbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:28.871100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-16921f9e-d891-4447-ab31-3bd8dee82bbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:28.871100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:28.871100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:28.871100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:28.871100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:28.871100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:28.871100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:28.871100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:28.871100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:28.871100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:28.871100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:28.871100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:28.871100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:28.871100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:28.871100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:28.871100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:28.872307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:28.872307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:28.872307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:28.872307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:28.872307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:28.872307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:28.872307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:28.872307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:28.872307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:28.872307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:28.872307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:28.872307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:28.872307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:28.872307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:28.872307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:28.872307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:28.873152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:28.873152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:28.873152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:28.873152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:28.873152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:28.873152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:28.873152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:28.873152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:28.873152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:28.873152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:28.873152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:28.873152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:28.873152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 283/564] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:28.873922 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16921f9e-d891-4447-ab31-3bd8dee82bbe None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:28.901057 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb160a-113e-4c7a-b158-90600ea7082f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:28.948163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac074ee9-385b-4e97-8a82-323877d633a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:28.948961 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac074ee9-385b-4e97-8a82-323877d633a3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:28.949332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac074ee9-385b-4e97-8a82-323877d633a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:28.949699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac074ee9-385b-4e97-8a82-323877d633a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:28.967722 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac074ee9-385b-4e97-8a82-323877d633a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:28.970221 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac074ee9-385b-4e97-8a82-323877d633a3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:28.977255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:28.997981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ac074ee9-385b-4e97-8a82-323877d633a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:28.997981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:28.997981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:28.997981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:28.997981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:28.997981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:28.997981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:28.997981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:28.997981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:28.997981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:28.997981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:28.997981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:28.997981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:28.997981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:28.997981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:28.997981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:28.999807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:28.999807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:28.999807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:28.999807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:28.999807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:28.999807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:28.999807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:28.999807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:28.999807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:28.999807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:28.999807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:28.999807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:28.999807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:28.999807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:28.999807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:28.999807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:28.999807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:29.002550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:29.002550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:29.002550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:29.002550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 282/565] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:29.002550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8e048-6187-443e-8647-2302fe37d104 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:29.002550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8e048-6187-443e-8647-2302fe37d104 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:29.003397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8e048-6187-443e-8647-2302fe37d104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:29.012437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16921f9e-d891-4447-ab31-3bd8dee82bbe None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:29.035898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-96a8e048-6187-443e-8647-2302fe37d104 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:29.038538 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-96a8e048-6187-443e-8647-2302fe37d104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:29.056431 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c2eb1-0ee2-4ebc-85c2-1de75f8d9685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:29.057078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c2eb1-0ee2-4ebc-85c2-1de75f8d9685 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:29.057355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c2eb1-0ee2-4ebc-85c2-1de75f8d9685 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:29.057624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c2eb1-0ee2-4ebc-85c2-1de75f8d9685 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:29.069987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c2eb1-0ee2-4ebc-85c2-1de75f8d9685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:29.071676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c2eb1-0ee2-4ebc-85c2-1de75f8d9685 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:29.093585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-b22c2eb1-0ee2-4ebc-85c2-1de75f8d9685 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:29.093585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:29.093585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:29.093585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:29.093585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:29.093585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:29.093585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:29.093585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:29.093585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:29.093585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:29.093585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:29.093585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:29.093585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:29.093585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:29.093585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:29.093585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:29.103377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:29.103377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:29.103377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:29.103377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:29.103377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:29.103377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:29.103377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:29.103377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:29.103377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:29.103377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:29.103377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:29.103377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:29.103377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:29.103377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:29.103377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:29.103377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:29.103377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:29.105009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:29.105009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:29.105009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:29.105009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 284/566] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:29.105009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c2eb1-0ee2-4ebc-85c2-1de75f8d9685 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:29.126559 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:29.132504 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 283/567] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:29.133619 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8e048-6187-443e-8647-2302fe37d104 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:29.258980 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8e048-6187-443e-8647-2302fe37d104 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:29.291672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c2eb1-0ee2-4ebc-85c2-1de75f8d9685 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:29.297715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9ea0b-59dc-48a8-a3f9-bbed1076452a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:29.298333 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9ea0b-59dc-48a8-a3f9-bbed1076452a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:29.298609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9ea0b-59dc-48a8-a3f9-bbed1076452a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:29.298891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9ea0b-59dc-48a8-a3f9-bbed1076452a None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:29.299522 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9ea0b-59dc-48a8-a3f9-bbed1076452a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:29.299823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9ea0b-59dc-48a8-a3f9-bbed1076452a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:29.301500 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9ea0b-59dc-48a8-a3f9-bbed1076452a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:29.316968 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 284/568] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:29.323623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9ea0b-59dc-48a8-a3f9-bbed1076452a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:29.354602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684022ff-a65e-4963-951b-628c7b5164de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:29.355219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684022ff-a65e-4963-951b-628c7b5164de None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:29.355498 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684022ff-a65e-4963-951b-628c7b5164de None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:29.355768 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684022ff-a65e-4963-951b-628c7b5164de None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:29.356473 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684022ff-a65e-4963-951b-628c7b5164de None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:29.356776 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684022ff-a65e-4963-951b-628c7b5164de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:29.358169 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684022ff-a65e-4963-951b-628c7b5164de None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:29.361164 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:29.369697 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 285/569] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:42:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:29.370059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684022ff-a65e-4963-951b-628c7b5164de None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:29.407732 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:29.451108 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9ea0b-59dc-48a8-a3f9-bbed1076452a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:29.480197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684022ff-a65e-4963-951b-628c7b5164de None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:29.487106 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9dce3-d667-423f-a01e-95f9d54010b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:29.487738 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9dce3-d667-423f-a01e-95f9d54010b7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:29.488021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9dce3-d667-423f-a01e-95f9d54010b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:29.488674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9dce3-d667-423f-a01e-95f9d54010b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:29.499216 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9dce3-d667-423f-a01e-95f9d54010b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:29.504311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-51b9dce3-d667-423f-a01e-95f9d54010b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:29.504311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:29.504311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:29.504311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:29.504311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:29.504311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:29.504311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:29.504311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:29.504311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:29.504311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:29.504311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:29.504311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:29.504311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:29.504311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:29.504311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:29.504311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:29.505262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:29.505262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:29.505262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:29.505262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:29.505262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:29.505262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:29.505262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:29.505262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:29.505262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:29.505262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:29.505262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:29.505262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:29.505262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:29.505262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:29.505262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:29.505262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:29.506104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:29.506104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:29.506104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:29.506104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:29.506104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:29.506104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:29.506104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:29.506104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:29.506104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:29.506104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:29.506104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:29.506104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:29.507363 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 285/570] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:29.508119 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9dce3-d667-423f-a01e-95f9d54010b7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:29.518273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5694b-4259-42d2-8387-d0005837871f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:29.518884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5694b-4259-42d2-8387-d0005837871f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:29.519166 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5694b-4259-42d2-8387-d0005837871f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:29.519436 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5694b-4259-42d2-8387-d0005837871f None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:29.520084 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5694b-4259-42d2-8387-d0005837871f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:29.520411 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5694b-4259-42d2-8387-d0005837871f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:29.521802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5694b-4259-42d2-8387-d0005837871f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:29.534129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 286/571] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:29.535075 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5694b-4259-42d2-8387-d0005837871f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:29.597183 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:29.620336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9dce3-d667-423f-a01e-95f9d54010b7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:29.639425 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5694b-4259-42d2-8387-d0005837871f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:29.640916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:29.656163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801aea46-7709-4736-bfa7-26d8009ad2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:29.656795 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801aea46-7709-4736-bfa7-26d8009ad2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:29.657079 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801aea46-7709-4736-bfa7-26d8009ad2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:29.657349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801aea46-7709-4736-bfa7-26d8009ad2b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:29.664163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801aea46-7709-4736-bfa7-26d8009ad2b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:29.665592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801aea46-7709-4736-bfa7-26d8009ad2b2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:29.673753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c9317-5d90-4b61-88f7-2695764bb4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:29.674379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c9317-5d90-4b61-88f7-2695764bb4ba None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:29.674653 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c9317-5d90-4b61-88f7-2695764bb4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:29.674921 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c9317-5d90-4b61-88f7-2695764bb4ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:29.678100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-801aea46-7709-4736-bfa7-26d8009ad2b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:29.678100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:29.678100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:29.678100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:29.678100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:29.678100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:29.678100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:29.678100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:29.678100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:29.678100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:29.678100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:29.678100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:29.678100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:29.678100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:29.678100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:29.684377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:29.684377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:29.684377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:29.684377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:29.684377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:29.684377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:29.684377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:29.684377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:29.684377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:29.684377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:29.684377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:29.684377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:29.684377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:29.684377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:29.684377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:29.684377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:29.684377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:29.685303 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:29.685303 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:29.685303 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:29.685303 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:29.685303 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:29.685303 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:29.685303 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:29.685303 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:29.685303 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 286/572] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:29.685303 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801aea46-7709-4736-bfa7-26d8009ad2b2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:29.686846 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c9317-5d90-4b61-88f7-2695764bb4ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:29.687562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c9317-5d90-4b61-88f7-2695764bb4ba None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:29.702575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-3d2c9317-5d90-4b61-88f7-2695764bb4ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:29.702575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:29.702575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:29.702575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:29.702575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:29.702575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:29.702575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:29.702575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:29.702575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:29.702575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:29.702575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:29.702575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:29.702575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:29.702575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:29.702575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:29.702575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:29.704067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:29.704067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:29.704067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:29.704067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:29.704067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:29.704067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:29.704067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:29.704067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:29.704067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:29.704067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:29.704067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:29.704067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:29.704067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:29.704067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:29.704067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:29.704067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:29.704067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:29.705493 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:29.705493 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:29.705493 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:29.705493 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 287/573] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:29.705493 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c9317-5d90-4b61-88f7-2695764bb4ba None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:29.787116 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801aea46-7709-4736-bfa7-26d8009ad2b2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:29.809322 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c9317-5d90-4b61-88f7-2695764bb4ba None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:29.821047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:29.822735 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57cedc-abc4-4f9c-9f07-f27471c30568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:29.822973 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57cedc-abc4-4f9c-9f07-f27471c30568 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:29.823296 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57cedc-abc4-4f9c-9f07-f27471c30568 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:29.823589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57cedc-abc4-4f9c-9f07-f27471c30568 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:29.835351 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57cedc-abc4-4f9c-9f07-f27471c30568 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:29.840766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-6f57cedc-abc4-4f9c-9f07-f27471c30568 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:29.840766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:29.840766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:29.840766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:29.840766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:29.840766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:29.840766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:29.840766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:29.840766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:29.840766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:29.840766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:29.840766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:29.840766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:29.840766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:29.840766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:29.840766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:29.841884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:29.841884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:29.841884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:29.841884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:29.841884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:29.841884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:29.841884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:29.841884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:29.841884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:29.841884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:29.841884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:29.841884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:29.841884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:29.841884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:29.841884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:29.841884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:29.842761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:29.842761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:29.842761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:29.842761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:29.842761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:29.842761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:29.842761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:29.842761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:29.842761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:29.842761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:29.842761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:29.842761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:29.842761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 287/574] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:29.842761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57cedc-abc4-4f9c-9f07-f27471c30568 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:29.849865 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b8f28-1613-4e2e-9a0f-19a9f777d68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:29.850516 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b8f28-1613-4e2e-9a0f-19a9f777d68a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:29.850794 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b8f28-1613-4e2e-9a0f-19a9f777d68a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:29.851125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b8f28-1613-4e2e-9a0f-19a9f777d68a None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:29.851858 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b8f28-1613-4e2e-9a0f-19a9f777d68a None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:29.852187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b8f28-1613-4e2e-9a0f-19a9f777d68a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:29.853561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b8f28-1613-4e2e-9a0f-19a9f777d68a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:29.862059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 288/575] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:42:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:29.863142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b8f28-1613-4e2e-9a0f-19a9f777d68a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:29.892638 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:29.953228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57cedc-abc4-4f9c-9f07-f27471c30568 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:29.965157 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b8f28-1613-4e2e-9a0f-19a9f777d68a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:29.986583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd4810-917f-4246-a338-dcf6b58ccec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:29.987218 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd4810-917f-4246-a338-dcf6b58ccec7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:29.987489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd4810-917f-4246-a338-dcf6b58ccec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:29.987752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd4810-917f-4246-a338-dcf6b58ccec7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:29.995368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd4810-917f-4246-a338-dcf6b58ccec7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:29.996858 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd4810-917f-4246-a338-dcf6b58ccec7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:30.000441 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777221ff-eb29-4c36-b9c3-247a64f17cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:30.001047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777221ff-eb29-4c36-b9c3-247a64f17cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:30.001327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777221ff-eb29-4c36-b9c3-247a64f17cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:30.001599 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777221ff-eb29-4c36-b9c3-247a64f17cc0 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:30.002292 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777221ff-eb29-4c36-b9c3-247a64f17cc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:30.002595 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777221ff-eb29-4c36-b9c3-247a64f17cc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:30.004003 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777221ff-eb29-4c36-b9c3-247a64f17cc0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:30.008120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-3dfd4810-917f-4246-a338-dcf6b58ccec7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:30.008120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:30.008120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:30.008120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:30.008120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:30.008120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:30.008120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:30.008120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:30.008120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:30.008120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:30.008120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:30.008120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:30.008120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:30.008120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:30.008120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:30.009162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:30.009162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:30.009162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:30.009162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:30.009162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:30.009162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:30.009162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:30.009162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:30.009162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:30.009162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:30.009162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:30.009162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:30.009162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:30.009162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:30.009162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:30.009162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:30.009162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:30.010080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:30.010080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:30.010080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:30.010080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:30.010080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:30.010080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:30.010080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:30.010080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:30.010080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 288/576] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:30.010578 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd4810-917f-4246-a338-dcf6b58ccec7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:30.016429 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 289/577] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:30.017246 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777221ff-eb29-4c36-b9c3-247a64f17cc0 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:30.048821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:30.113427 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:30.116858 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd4810-917f-4246-a338-dcf6b58ccec7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:30.119332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777221ff-eb29-4c36-b9c3-247a64f17cc0 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:30.151499 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf696d01-7d16-4d26-9365-0ed81570f740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:30.152128 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf696d01-7d16-4d26-9365-0ed81570f740 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:30.152417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf696d01-7d16-4d26-9365-0ed81570f740 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:30.152686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf696d01-7d16-4d26-9365-0ed81570f740 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:30.153326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf696d01-7d16-4d26-9365-0ed81570f740 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:30.153628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf696d01-7d16-4d26-9365-0ed81570f740 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:30.154985 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf696d01-7d16-4d26-9365-0ed81570f740 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:30.155713 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d000d-d24c-411c-b81f-a6ad002fa35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:30.156358 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d000d-d24c-411c-b81f-a6ad002fa35f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:30.156641 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d000d-d24c-411c-b81f-a6ad002fa35f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:30.156908 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d000d-d24c-411c-b81f-a6ad002fa35f None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:30.157571 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d000d-d24c-411c-b81f-a6ad002fa35f None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:30.157873 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d000d-d24c-411c-b81f-a6ad002fa35f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:30.159229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d000d-d24c-411c-b81f-a6ad002fa35f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:30.167817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 289/578] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:30.172901 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf696d01-7d16-4d26-9365-0ed81570f740 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:30.173057 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 290/579] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:42:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:30.173057 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d000d-d24c-411c-b81f-a6ad002fa35f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:30.266633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:30.273731 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf696d01-7d16-4d26-9365-0ed81570f740 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:30.274462 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d000d-d24c-411c-b81f-a6ad002fa35f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:30.309555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c1606-4978-4caf-a316-1a33465e3b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:30.310188 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c1606-4978-4caf-a316-1a33465e3b2c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:30.310466 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c1606-4978-4caf-a316-1a33465e3b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:30.310732 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c1606-4978-4caf-a316-1a33465e3b2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:30.320223 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001303a-c8fe-479c-8c35-34680d095be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:30.320842 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001303a-c8fe-479c-8c35-34680d095be8 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:30.321117 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001303a-c8fe-479c-8c35-34680d095be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:30.321386 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001303a-c8fe-479c-8c35-34680d095be8 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:30.321890 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c1606-4978-4caf-a316-1a33465e3b2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:30.322086 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001303a-c8fe-479c-8c35-34680d095be8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:30.322433 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001303a-c8fe-479c-8c35-34680d095be8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:30.323382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c1606-4978-4caf-a316-1a33465e3b2c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:30.323809 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001303a-c8fe-479c-8c35-34680d095be8 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:30.337263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 291/580] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:30.339050 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001303a-c8fe-479c-8c35-34680d095be8 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:30.340287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-147c1606-4978-4caf-a316-1a33465e3b2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:30.340287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:30.340287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:30.340287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:30.340287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:30.340287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:30.340287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:30.340287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:30.340287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:30.340287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:30.340287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:30.340287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:30.340287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:30.340287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:30.340287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:30.340287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:30.344509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:30.344509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:30.344509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:30.344509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:30.344509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:30.344509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:30.344509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:30.344509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:30.344509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:30.344509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:30.344509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:30.344509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:30.344509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:30.344509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:30.344509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:30.344509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:30.344509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:30.345685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:30.345685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:30.345685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:30.345685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 290/581] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:30.345685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c1606-4978-4caf-a316-1a33465e3b2c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:30.369121 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:30.451730 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001303a-c8fe-479c-8c35-34680d095be8 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:30.465269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c1606-4978-4caf-a316-1a33465e3b2c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:30.488164 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cf2b9-d23b-4e09-bc5d-a2c003846ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:30.488839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cf2b9-d23b-4e09-bc5d-a2c003846ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:30.489117 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cf2b9-d23b-4e09-bc5d-a2c003846ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:30.489392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cf2b9-d23b-4e09-bc5d-a2c003846ae1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:30.490118 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6cf2b9-d23b-4e09-bc5d-a2c003846ae1 None admin] RBAC: Adding query filter params (role.id=9aa748c8ea2542e6987e0b93190ea4e6, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=46b1417d4ac24c8982b2052f4e99042a) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:30.490422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6cf2b9-d23b-4e09-bc5d-a2c003846ae1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:30.492027 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6cf2b9-d23b-4e09-bc5d-a2c003846ae1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:30.497634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 292/582] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:42:30 2022] GET /identity/v3/role_assignments?user.id=46b1417d4ac24c8982b2052f4e99042a&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=9aa748c8ea2542e6987e0b93190ea4e6 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:30.498526 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cf2b9-d23b-4e09-bc5d-a2c003846ae1 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:30.499095 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd49763-c111-435c-81df-526c333c5a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:30.499843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd49763-c111-435c-81df-526c333c5a25 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:30.500142 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd49763-c111-435c-81df-526c333c5a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:30.500414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd49763-c111-435c-81df-526c333c5a25 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:30.510168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd49763-c111-435c-81df-526c333c5a25 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:30.512235 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd49763-c111-435c-81df-526c333c5a25 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:30.521051 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:30.529361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-1cd49763-c111-435c-81df-526c333c5a25 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 08:42:30.529361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:30.529361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:30.529361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:30.529361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:30.529361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:30.529361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:30.529361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:30.529361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:30.529361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:30.529361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:30.529361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:30.529361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:30.529361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:30.529361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:30.529361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:30.530563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:30.530563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:30.530563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:30.530563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:30.530563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:30.530563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:30.530563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:30.530563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:30.530563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:30.530563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:30.530563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:30.530563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:30.530563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:30.530563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:30.530563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:30.530563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:30.530563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:30.530563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:30.531540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:30.531540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:30.531540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:30.531540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:30.531540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:30.531540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:30.531540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:30.531540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:30.531540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 08:42:30.531540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:30.531540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 291/583] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:42:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:30.532196 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd49763-c111-435c-81df-526c333c5a25 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:30.619948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cf2b9-d23b-4e09-bc5d-a2c003846ae1 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:30.624787 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:30.661047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df3bf5-a558-425a-89ec-5e264809d282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:30.661713 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df3bf5-a558-425a-89ec-5e264809d282 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:30.661988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df3bf5-a558-425a-89ec-5e264809d282 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:30.662088 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd49763-c111-435c-81df-526c333c5a25 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:30.662324 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df3bf5-a558-425a-89ec-5e264809d282 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:30.675935 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df3bf5-a558-425a-89ec-5e264809d282 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:30.677466 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df3bf5-a558-425a-89ec-5e264809d282 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:30.693094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-54df3bf5-a558-425a-89ec-5e264809d282 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:30.693094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:30.693094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:30.693094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:30.693094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:30.693094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:30.693094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:30.693094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:30.693094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:30.693094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:30.693094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:30.693094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:30.693094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:30.693094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:30.693094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:30.693094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:30.694303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:30.694303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:30.694303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:30.694303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:30.694303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:30.694303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:30.694303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:30.694303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:30.694303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:30.694303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:30.694303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:30.694303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:30.694303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:30.694303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:30.694303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:30.694303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:30.694303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:30.695333 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:30.695333 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:30.695333 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:30.695333 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 293/584] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:30.695566 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df3bf5-a558-425a-89ec-5e264809d282 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:30.706256 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f055b9-de13-48f4-8d9f-e8dbd51743a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:30.707215 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f055b9-de13-48f4-8d9f-e8dbd51743a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:30.707701 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f055b9-de13-48f4-8d9f-e8dbd51743a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:30.708260 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f055b9-de13-48f4-8d9f-e8dbd51743a2 None admin] PATH_INFO: `/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/ca56dd7216fb4946946b2801e144fec2/roles/9aa748c8ea2542e6987e0b93190ea4e6` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:30.729219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f055b9-de13-48f4-8d9f-e8dbd51743a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=fce6ee19f58b4a468c9a5d169e29a018, user_id=ca56dd7216fb4946946b2801e144fec2, role_id=9aa748c8ea2542e6987e0b93190ea4e6)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:30.731551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f055b9-de13-48f4-8d9f-e8dbd51743a2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:30.754555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 292/585] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:42:30 2022] PUT /identity/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/ca56dd7216fb4946946b2801e144fec2/roles/9aa748c8ea2542e6987e0b93190ea4e6 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:42:30.765195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f055b9-de13-48f4-8d9f-e8dbd51743a2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:30.788044 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:30.837404 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df3bf5-a558-425a-89ec-5e264809d282 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:30.883389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43471c9-d8c3-45e7-b597-9f879ff1ce0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:30.884079 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43471c9-d8c3-45e7-b597-9f879ff1ce0f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:30.884488 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43471c9-d8c3-45e7-b597-9f879ff1ce0f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:30.884863 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43471c9-d8c3-45e7-b597-9f879ff1ce0f None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:30.885660 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43471c9-d8c3-45e7-b597-9f879ff1ce0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:30.886083 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43471c9-d8c3-45e7-b597-9f879ff1ce0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:30.887639 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43471c9-d8c3-45e7-b597-9f879ff1ce0f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:30.909371 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 294/586] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:30.909371 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43471c9-d8c3-45e7-b597-9f879ff1ce0f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:30.935134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f055b9-de13-48f4-8d9f-e8dbd51743a2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:30.966625 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:30.980323 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba7845-de05-4970-89e8-a7af7c7be7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:30.980949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba7845-de05-4970-89e8-a7af7c7be7a6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:30.981232 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba7845-de05-4970-89e8-a7af7c7be7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:30.981504 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba7845-de05-4970-89e8-a7af7c7be7a6 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:30.982360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba7845-de05-4970-89e8-a7af7c7be7a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:30.982670 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba7845-de05-4970-89e8-a7af7c7be7a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:30.984284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba7845-de05-4970-89e8-a7af7c7be7a6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:31.005741 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 293/587] 10.222.0.67 () {56 vars in 1137 bytes} [Wed Aug 24 08:42:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:31.007187 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba7845-de05-4970-89e8-a7af7c7be7a6 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:31.058274 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43471c9-d8c3-45e7-b597-9f879ff1ce0f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:31.097854 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:31.104758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abe3b3-2813-4f58-82dd-c3c812802959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:31.106310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abe3b3-2813-4f58-82dd-c3c812802959 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:31.106795 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abe3b3-2813-4f58-82dd-c3c812802959 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:31.107246 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abe3b3-2813-4f58-82dd-c3c812802959 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:31.108393 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abe3b3-2813-4f58-82dd-c3c812802959 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:31.108904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abe3b3-2813-4f58-82dd-c3c812802959 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:31.111314 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abe3b3-2813-4f58-82dd-c3c812802959 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:31.128162 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 295/588] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:31.129664 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abe3b3-2813-4f58-82dd-c3c812802959 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:31.187025 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba7845-de05-4970-89e8-a7af7c7be7a6 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:31.232230 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca04862-140f-4fc7-9d9b-1e30aa9fe27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:31.233303 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca04862-140f-4fc7-9d9b-1e30aa9fe27f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:31.233786 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca04862-140f-4fc7-9d9b-1e30aa9fe27f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:31.234231 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca04862-140f-4fc7-9d9b-1e30aa9fe27f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:31.244110 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca04862-140f-4fc7-9d9b-1e30aa9fe27f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:31.245986 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca04862-140f-4fc7-9d9b-1e30aa9fe27f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:31.261198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-eca04862-140f-4fc7-9d9b-1e30aa9fe27f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:42:31.261198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:31.261198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:31.261198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:31.261198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:31.261198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:31.261198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:31.261198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:31.261198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:31.261198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:31.261198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:31.261198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:31.261198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:31.261198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:31.261198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:31.261198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:31.262193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:31.262193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:31.262193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:31.262193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:31.262193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:31.262193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:31.262193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:31.262193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:31.262193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:31.262193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:31.262193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:31.262193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:31.262193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:31.262193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:31.262193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:31.262193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:31.262193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:31.262193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:31.263140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:31.263140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:31.263140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:31.263140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:31.263140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:31.263140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:31.263140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:31.263140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:31.263140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:42:31.263140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:31.270609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 294/589] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:42:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:31.270609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca04862-140f-4fc7-9d9b-1e30aa9fe27f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:31.288883 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abe3b3-2813-4f58-82dd-c3c812802959 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:31.333360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:31.337458 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3fbe8-8c6d-4400-950f-a8774f8e877b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:31.338903 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3fbe8-8c6d-4400-950f-a8774f8e877b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:31.339778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3fbe8-8c6d-4400-950f-a8774f8e877b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:31.340505 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3fbe8-8c6d-4400-950f-a8774f8e877b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:31.357914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3fbe8-8c6d-4400-950f-a8774f8e877b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:31.360409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3fbe8-8c6d-4400-950f-a8774f8e877b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:31.381695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-7fc3fbe8-8c6d-4400-950f-a8774f8e877b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:31.381695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:31.381695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:31.381695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:31.381695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:31.381695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:31.381695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:31.381695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:31.381695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:31.381695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:31.381695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:31.381695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:31.381695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:31.381695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:31.381695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:31.381695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:31.382792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:31.382792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:31.382792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:31.382792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:31.382792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:31.382792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:31.382792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:31.382792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:31.382792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:31.382792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:31.382792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:31.382792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:31.382792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:31.382792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:31.382792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:31.382792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:31.382792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:31.383680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:31.383680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:31.383680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:31.383680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 296/590] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:31.384378 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3fbe8-8c6d-4400-950f-a8774f8e877b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:31.422536 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca04862-140f-4fc7-9d9b-1e30aa9fe27f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:31.461327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94083f-10b0-4951-84ef-d711f3696b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:31.462008 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94083f-10b0-4951-84ef-d711f3696b44 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:31.462293 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94083f-10b0-4951-84ef-d711f3696b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:31.462565 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94083f-10b0-4951-84ef-d711f3696b44 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:31.471510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94083f-10b0-4951-84ef-d711f3696b44 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:31.473102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94083f-10b0-4951-84ef-d711f3696b44 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:31.476535 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:31.486704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-fb94083f-10b0-4951-84ef-d711f3696b44 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 08:42:31.486704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:31.486704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:31.486704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:31.486704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:31.486704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:31.486704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:31.486704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:31.486704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:31.486704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:31.486704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:31.486704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:31.486704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:31.486704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:31.486704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:31.486704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:31.487715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:31.487715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:31.487715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:31.487715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:31.487715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:31.487715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:31.487715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:31.487715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:31.487715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:31.487715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:31.487715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:31.487715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:31.487715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:31.487715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:31.487715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:31.487715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:31.487715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:31.487715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:31.488699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:31.488699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:31.488699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:31.488699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:31.488699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:31.488699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:31.488699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:31.488699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:31.488699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 08:42:31.488699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:31.496432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 295/591] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:31.497669 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94083f-10b0-4951-84ef-d711f3696b44 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:31.516288 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3fbe8-8c6d-4400-950f-a8774f8e877b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:31.563756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0a8a8-cf4c-49b3-bac0-bf7394c0bccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:31.564407 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0a8a8-cf4c-49b3-bac0-bf7394c0bccc None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:31.564904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0a8a8-cf4c-49b3-bac0-bf7394c0bccc None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:31.565191 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0a8a8-cf4c-49b3-bac0-bf7394c0bccc None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:31.565922 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0a8a8-cf4c-49b3-bac0-bf7394c0bccc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:31.566631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0a8a8-cf4c-49b3-bac0-bf7394c0bccc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:31.568385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0a8a8-cf4c-49b3-bac0-bf7394c0bccc None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:31.594338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 297/592] 10.222.0.67 () {56 vars in 1137 bytes} [Wed Aug 24 08:42:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:31.596361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0a8a8-cf4c-49b3-bac0-bf7394c0bccc None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:31.622097 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:31.626501 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94083f-10b0-4951-84ef-d711f3696b44 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:31.666260 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d296e1-3441-40e1-9459-40265a60cfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:31.667243 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d296e1-3441-40e1-9459-40265a60cfac None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:31.667757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d296e1-3441-40e1-9459-40265a60cfac None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:31.668268 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d296e1-3441-40e1-9459-40265a60cfac None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:31.669135 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d296e1-3441-40e1-9459-40265a60cfac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:31.669700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d296e1-3441-40e1-9459-40265a60cfac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:31.671341 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d296e1-3441-40e1-9459-40265a60cfac None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:31.685919 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 296/593] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:31.687164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d296e1-3441-40e1-9459-40265a60cfac None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:31.731636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:31.737603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0a8a8-cf4c-49b3-bac0-bf7394c0bccc None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:31.783461 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0b2ad-01d7-4fb1-aace-62825c6a5214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:31.784499 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0b2ad-01d7-4fb1-aace-62825c6a5214 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:31.784986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0b2ad-01d7-4fb1-aace-62825c6a5214 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:31.785460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0b2ad-01d7-4fb1-aace-62825c6a5214 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:31.786395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0b2ad-01d7-4fb1-aace-62825c6a5214 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:31.786949 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0b2ad-01d7-4fb1-aace-62825c6a5214 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:31.789433 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0b2ad-01d7-4fb1-aace-62825c6a5214 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:31.813768 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 298/594] 10.222.0.67 () {56 vars in 1139 bytes} [Wed Aug 24 08:42:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:31.815046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0b2ad-01d7-4fb1-aace-62825c6a5214 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:31.815705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d296e1-3441-40e1-9459-40265a60cfac None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:31.861045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead65191-2ed3-49c3-bee9-b96f80f58bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:31.862154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead65191-2ed3-49c3-bee9-b96f80f58bfd None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:31.862642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead65191-2ed3-49c3-bee9-b96f80f58bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:31.863199 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead65191-2ed3-49c3-bee9-b96f80f58bfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:31.876014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead65191-2ed3-49c3-bee9-b96f80f58bfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:31.878053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead65191-2ed3-49c3-bee9-b96f80f58bfd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:31.895421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ead65191-2ed3-49c3-bee9-b96f80f58bfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:31.895421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:31.895421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:31.895421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:31.895421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:31.895421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:31.895421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:31.895421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:31.895421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:31.895421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:31.895421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:31.895421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:31.895421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:31.895421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:31.895421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:31.895421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:31.896462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:31.896462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:31.896462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:31.896462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:31.896462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:31.896462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:31.896462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:31.896462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:31.896462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:31.896462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:31.896462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:31.896462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:31.896462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:31.896462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:31.896462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:31.896462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:31.896462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:31.897511 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:31.897511 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:31.897511 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:31.903931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 297/595] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:31.903931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead65191-2ed3-49c3-bee9-b96f80f58bfd None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:31.924378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:31.944702 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0b2ad-01d7-4fb1-aace-62825c6a5214 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:31.992854 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d88bfc-6e72-4ff1-9df1-92f27447aa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:31.993842 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d88bfc-6e72-4ff1-9df1-92f27447aa97 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:31.994319 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d88bfc-6e72-4ff1-9df1-92f27447aa97 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:31.994807 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d88bfc-6e72-4ff1-9df1-92f27447aa97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:32.004072 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d88bfc-6e72-4ff1-9df1-92f27447aa97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:32.007168 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d88bfc-6e72-4ff1-9df1-92f27447aa97 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:32.024427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-68d88bfc-6e72-4ff1-9df1-92f27447aa97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:32.024427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:32.024427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:32.024427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:32.024427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:32.024427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:32.024427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:32.024427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:32.024427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:32.024427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:32.024427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:32.024427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:32.024427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:32.024427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:32.024427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:32.025380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:32.025380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:32.025380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:32.025380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:32.025380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:32.025380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:32.025380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:32.025380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:32.025380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:32.025380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:32.025380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:32.025380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:32.025380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:32.025380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:32.025380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:32.025380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:32.025380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:32.026256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:32.026256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:32.026256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:32.026256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:32.026256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:32.026256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:32.026256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:32.026256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:32.026680 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead65191-2ed3-49c3-bee9-b96f80f58bfd None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:32.029869 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 299/596] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:32.030910 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d88bfc-6e72-4ff1-9df1-92f27447aa97 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:32.057174 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:32.074557 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94170772-09be-47c0-92a5-d67ca57c1a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:32.076038 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94170772-09be-47c0-92a5-d67ca57c1a48 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:32.076582 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94170772-09be-47c0-92a5-d67ca57c1a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:32.077057 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94170772-09be-47c0-92a5-d67ca57c1a48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:32.092783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94170772-09be-47c0-92a5-d67ca57c1a48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:32.094410 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94170772-09be-47c0-92a5-d67ca57c1a48 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:32.109993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-94170772-09be-47c0-92a5-d67ca57c1a48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:32.109993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:32.109993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:32.109993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:32.109993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:32.109993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:32.109993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:32.109993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:32.109993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:32.109993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:32.109993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:32.109993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:32.109993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:32.109993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:32.109993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:32.109993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:32.111254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:32.111254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:32.111254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:32.111254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:32.111254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:32.111254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:32.111254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:32.111254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:32.111254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:32.111254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:32.111254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:32.111254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:32.111254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:32.111254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:32.111254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:32.111254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:32.111254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:32.112401 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:32.112401 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:32.112401 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:32.118847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 298/597] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:32.119867 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94170772-09be-47c0-92a5-d67ca57c1a48 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:32.160153 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d88bfc-6e72-4ff1-9df1-92f27447aa97 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:32.189023 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:32.200545 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3e7ec-781d-4c92-8ae8-02492480d2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:32.201451 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3e7ec-781d-4c92-8ae8-02492480d2b4 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:32.201882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3e7ec-781d-4c92-8ae8-02492480d2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:32.202300 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3e7ec-781d-4c92-8ae8-02492480d2b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:32.203098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3e7ec-781d-4c92-8ae8-02492480d2b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:32.203551 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3e7ec-781d-4c92-8ae8-02492480d2b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:32.205115 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3e7ec-781d-4c92-8ae8-02492480d2b4 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:32.222239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 300/598] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:32.223309 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3e7ec-781d-4c92-8ae8-02492480d2b4 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:32.236328 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94170772-09be-47c0-92a5-d67ca57c1a48 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:32.283784 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aabdf1-c936-46c4-97a9-cc758397c7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:32.284971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aabdf1-c936-46c4-97a9-cc758397c7cd None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:32.285407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aabdf1-c936-46c4-97a9-cc758397c7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:32.285852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aabdf1-c936-46c4-97a9-cc758397c7cd None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:32.287112 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aabdf1-c936-46c4-97a9-cc758397c7cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:32.287603 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aabdf1-c936-46c4-97a9-cc758397c7cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:32.289344 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aabdf1-c936-46c4-97a9-cc758397c7cd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:32.308226 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 299/599] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:32.309311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aabdf1-c936-46c4-97a9-cc758397c7cd None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:32.328575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:32.353908 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3e7ec-781d-4c92-8ae8-02492480d2b4 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:32.405196 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd756ae-e4b6-4963-b6bd-64f3526bba75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:32.406090 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd756ae-e4b6-4963-b6bd-64f3526bba75 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:32.406549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd756ae-e4b6-4963-b6bd-64f3526bba75 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:32.407694 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd756ae-e4b6-4963-b6bd-64f3526bba75 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:32.408627 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd756ae-e4b6-4963-b6bd-64f3526bba75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:32.409122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd756ae-e4b6-4963-b6bd-64f3526bba75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:32.411391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd756ae-e4b6-4963-b6bd-64f3526bba75 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:32.431091 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 301/600] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:32.432211 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd756ae-e4b6-4963-b6bd-64f3526bba75 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:32.447205 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aabdf1-c936-46c4-97a9-cc758397c7cd None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:32.478324 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:32.490506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605122f-da1a-4dc0-b562-1ea56e76c0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:32.491193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605122f-da1a-4dc0-b562-1ea56e76c0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:32.491703 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605122f-da1a-4dc0-b562-1ea56e76c0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:32.491981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605122f-da1a-4dc0-b562-1ea56e76c0d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:32.499224 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d605122f-da1a-4dc0-b562-1ea56e76c0d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:32.500714 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d605122f-da1a-4dc0-b562-1ea56e76c0d4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:32.513127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d605122f-da1a-4dc0-b562-1ea56e76c0d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:32.513127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:32.513127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:32.513127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:32.513127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:32.513127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:32.513127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:32.513127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:32.513127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:32.513127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:32.513127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:32.513127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:32.513127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:32.513127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:32.513127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:32.514191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:32.514191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:32.514191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:32.514191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:32.514191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:32.514191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:32.514191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:32.514191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:32.514191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:32.514191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:32.514191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:32.514191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:32.514191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:32.514191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:32.514191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:32.514191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:32.514191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:32.515287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:32.515287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:32.515287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:32.515287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:32.515287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:32.515287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:32.515287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:32.515287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:32.515287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 300/601] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:32.515287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605122f-da1a-4dc0-b562-1ea56e76c0d4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:32.556402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd756ae-e4b6-4963-b6bd-64f3526bba75 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:32.593878 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39d84a-ca08-4745-86da-6c97d2a85dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:32.594534 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39d84a-ca08-4745-86da-6c97d2a85dfa None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:32.594812 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39d84a-ca08-4745-86da-6c97d2a85dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:32.595075 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39d84a-ca08-4745-86da-6c97d2a85dfa None admin] PATH_INFO: `/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/5979387f7aa54619b8d429368c306cc9/roles/9aa748c8ea2542e6987e0b93190ea4e6` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:32.603943 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:32.612601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39d84a-ca08-4745-86da-6c97d2a85dfa None admin] RBAC: Authorizing `identity:create_grant(project_id=fce6ee19f58b4a468c9a5d169e29a018, user_id=5979387f7aa54619b8d429368c306cc9, role_id=9aa748c8ea2542e6987e0b93190ea4e6)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:32.614034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39d84a-ca08-4745-86da-6c97d2a85dfa None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:32.628222 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605122f-da1a-4dc0-b562-1ea56e76c0d4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:32.628799 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 302/602] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:42:32 2022] PUT /identity/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/5979387f7aa54619b8d429368c306cc9/roles/9aa748c8ea2542e6987e0b93190ea4e6 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:42:32.629965 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307ae09-b5c7-40cb-8f81-0c49a1812e3a None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:32.630268 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307ae09-b5c7-40cb-8f81-0c49a1812e3a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:32.630526 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307ae09-b5c7-40cb-8f81-0c49a1812e3a None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:32.636232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 303/603] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:32.636232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307ae09-b5c7-40cb-8f81-0c49a1812e3a None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:32.683162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf36192-3407-4965-adfa-85276c9183fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:32.684184 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf36192-3407-4965-adfa-85276c9183fa None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:32.684639 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf36192-3407-4965-adfa-85276c9183fa None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:32.685049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf36192-3407-4965-adfa-85276c9183fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:32.693777 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf36192-3407-4965-adfa-85276c9183fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:32.696761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf36192-3407-4965-adfa-85276c9183fa None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:32.711395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-bdf36192-3407-4965-adfa-85276c9183fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:32.711395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:32.711395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:32.711395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:32.711395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:32.711395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:32.711395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:32.711395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:32.711395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:32.711395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:32.711395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:32.711395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:32.711395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:32.711395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:32.711395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:32.712613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:32.712613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:32.712613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:32.712613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:32.712613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:32.712613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:32.712613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:32.712613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:32.712613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:32.712613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:32.712613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:32.712613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:32.712613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:32.712613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:32.712613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:32.712613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:32.712613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:32.713510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:32.713510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:32.713510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:32.713510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:32.713510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:32.713510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:32.713510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:32.713510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:32.716056 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 301/604] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:32.719748 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f57d2-7231-45e6-96b2-b696860d97c1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:32.720290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f57d2-7231-45e6-96b2-b696860d97c1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:32.720759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f57d2-7231-45e6-96b2-b696860d97c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:32.755071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5e4f57d2-7231-45e6-96b2-b696860d97c1 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:32.758125 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5e4f57d2-7231-45e6-96b2-b696860d97c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:32.788743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307ae09-b5c7-40cb-8f81-0c49a1812e3a None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:32.819673 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:32.836745 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fde3a-e445-42d6-beba-2c0b2efc4d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:32.837364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fde3a-e445-42d6-beba-2c0b2efc4d61 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:32.837677 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fde3a-e445-42d6-beba-2c0b2efc4d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:32.837931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fde3a-e445-42d6-beba-2c0b2efc4d61 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:32.838595 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fde3a-e445-42d6-beba-2c0b2efc4d61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:32.838900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fde3a-e445-42d6-beba-2c0b2efc4d61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:32.840635 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fde3a-e445-42d6-beba-2c0b2efc4d61 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:32.852499 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 302/605] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:32.853765 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f57d2-7231-45e6-96b2-b696860d97c1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:32.854287 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 304/606] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:32.859188 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca0bfe-ceba-4bb6-a80c-3f954fee7f16 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:32.859188 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca0bfe-ceba-4bb6-a80c-3f954fee7f16 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:32.859188 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca0bfe-ceba-4bb6-a80c-3f954fee7f16 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:32.859188 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 305/607] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:32.860164 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca0bfe-ceba-4bb6-a80c-3f954fee7f16 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:32.921094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:32.983158 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f57d2-7231-45e6-96b2-b696860d97c1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:32.996031 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca0bfe-ceba-4bb6-a80c-3f954fee7f16 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:33.022735 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b36dd-3516-44d4-b298-647f86b9561a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:33.023791 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b36dd-3516-44d4-b298-647f86b9561a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:33.024644 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b36dd-3516-44d4-b298-647f86b9561a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:33.025453 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b36dd-3516-44d4-b298-647f86b9561a None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:33.026928 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b36dd-3516-44d4-b298-647f86b9561a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:33.027845 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b36dd-3516-44d4-b298-647f86b9561a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:33.029799 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b36dd-3516-44d4-b298-647f86b9561a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:33.036514 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524f787-ed69-4643-80c1-69b351569d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:33.037170 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524f787-ed69-4643-80c1-69b351569d12 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:33.037442 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524f787-ed69-4643-80c1-69b351569d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:33.037731 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524f787-ed69-4643-80c1-69b351569d12 None admin] PATH_INFO: `/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/5de1586efc8c4dec898b0fb54b56acf0/roles/9aa748c8ea2542e6987e0b93190ea4e6` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:33.045269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 303/608] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:33.047004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165d382-62d8-4cbc-8c02-919de07b402f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:33.047498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165d382-62d8-4cbc-8c02-919de07b402f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:33.047999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165d382-62d8-4cbc-8c02-919de07b402f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:33.060583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524f787-ed69-4643-80c1-69b351569d12 None admin] RBAC: Authorizing `identity:create_grant(project_id=fce6ee19f58b4a468c9a5d169e29a018, user_id=5de1586efc8c4dec898b0fb54b56acf0, role_id=9aa748c8ea2542e6987e0b93190ea4e6)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:33.063106 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524f787-ed69-4643-80c1-69b351569d12 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:33.076726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8165d382-62d8-4cbc-8c02-919de07b402f None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:33.078103 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8165d382-62d8-4cbc-8c02-919de07b402f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:33.079653 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 306/609] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:42:32 2022] PUT /identity/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/5de1586efc8c4dec898b0fb54b56acf0/roles/9aa748c8ea2542e6987e0b93190ea4e6 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:42:33.080902 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524f787-ed69-4643-80c1-69b351569d12 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:33.113022 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:33.184004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 304/610] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:33.185888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165d382-62d8-4cbc-8c02-919de07b402f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:33.222655 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524f787-ed69-4643-80c1-69b351569d12 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:33.236663 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:33.269824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcf392-9633-4906-aa0b-54ee5f288193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:33.270957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcf392-9633-4906-aa0b-54ee5f288193 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:33.271447 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcf392-9633-4906-aa0b-54ee5f288193 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:33.271912 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcf392-9633-4906-aa0b-54ee5f288193 None admin] PATH_INFO: `/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/2ee2e93901914b1d82607a3ae0e2acf1/roles/9aa748c8ea2542e6987e0b93190ea4e6` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:33.292638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffcf392-9633-4906-aa0b-54ee5f288193 None admin] RBAC: Authorizing `identity:create_grant(project_id=fce6ee19f58b4a468c9a5d169e29a018, user_id=2ee2e93901914b1d82607a3ae0e2acf1, role_id=9aa748c8ea2542e6987e0b93190ea4e6)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:33.293907 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffcf392-9633-4906-aa0b-54ee5f288193 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:33.320136 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 307/611] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:42:33 2022] PUT /identity/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/2ee2e93901914b1d82607a3ae0e2acf1/roles/9aa748c8ea2542e6987e0b93190ea4e6 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:42:33.344011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165d382-62d8-4cbc-8c02-919de07b402f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:33.399081 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e0b11-fe17-4fd2-8b06-052ca87b9276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:33.400042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e0b11-fe17-4fd2-8b06-052ca87b9276 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:33.400508 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e0b11-fe17-4fd2-8b06-052ca87b9276 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:33.400962 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e0b11-fe17-4fd2-8b06-052ca87b9276 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:33.414096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e0b11-fe17-4fd2-8b06-052ca87b9276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:33.419462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d99e0b11-fe17-4fd2-8b06-052ca87b9276 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:33.419462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:33.419462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:33.419462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:33.419462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:33.419462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:33.419462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:33.419462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:33.419462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:33.419462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:33.419462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:33.419462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:33.419462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:33.419462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:33.419462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:33.419462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:33.420695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:33.420695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:33.420695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:33.420695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:33.420695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:33.420695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:33.420695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:33.420695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:33.420695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:33.420695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:33.420695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:33.420695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:33.420695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:33.420695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:33.420695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:33.420695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:33.421759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:33.421759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:33.421759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:33.421759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:33.421759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:33.421759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:33.421759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:33.421759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:33.421759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:33.421759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:33.421759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:33.421759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:33.424341 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 305/612] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:33.430702 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcf392-9633-4906-aa0b-54ee5f288193 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:33.558656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcf392-9633-4906-aa0b-54ee5f288193 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:33.595397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76052f33-bc1f-491d-a873-bbdee6130f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:33.596016 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76052f33-bc1f-491d-a873-bbdee6130f18 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:33.596323 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76052f33-bc1f-491d-a873-bbdee6130f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:33.596595 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76052f33-bc1f-491d-a873-bbdee6130f18 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:33.597266 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76052f33-bc1f-491d-a873-bbdee6130f18 None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:33.597565 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76052f33-bc1f-491d-a873-bbdee6130f18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:33.598933 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76052f33-bc1f-491d-a873-bbdee6130f18 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:33.608514 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e0b11-fe17-4fd2-8b06-052ca87b9276 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:33.608766 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 308/613] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:42:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:33.614158 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76052f33-bc1f-491d-a873-bbdee6130f18 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:33.629340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:33.654756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:33.776003 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76052f33-bc1f-491d-a873-bbdee6130f18 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:33.822152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e0b11-fe17-4fd2-8b06-052ca87b9276 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:33.869299 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be79b13-d38c-4538-a7ed-a3aeeb9b0803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:33.870426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be79b13-d38c-4538-a7ed-a3aeeb9b0803 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:33.870709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be79b13-d38c-4538-a7ed-a3aeeb9b0803 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:33.870983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be79b13-d38c-4538-a7ed-a3aeeb9b0803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:33.873747 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa1179-2fa1-4077-a0cc-d6f02300a0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:33.874391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa1179-2fa1-4077-a0cc-d6f02300a0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:33.874666 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa1179-2fa1-4077-a0cc-d6f02300a0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:33.874934 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa1179-2fa1-4077-a0cc-d6f02300a0e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:33.895956 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be79b13-d38c-4538-a7ed-a3aeeb9b0803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:33.898057 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be79b13-d38c-4538-a7ed-a3aeeb9b0803 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:33.899070 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa1179-2fa1-4077-a0cc-d6f02300a0e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:33.904164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-04fa1179-2fa1-4077-a0cc-d6f02300a0e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:33.904164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:33.904164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:33.904164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:33.904164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:33.904164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:33.904164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:33.904164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:33.904164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:33.904164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:33.904164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:33.904164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:33.904164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:33.904164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:33.904164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:33.904164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:33.905109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:33.905109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:33.905109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:33.905109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:33.905109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:33.905109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:33.905109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:33.905109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:33.905109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:33.905109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:33.905109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:33.905109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:33.905109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:33.905109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:33.905109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:33.905109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:33.905951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:33.905951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:33.905951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:33.905951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:33.905951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:33.905951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:33.905951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:33.905951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:33.905951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:33.905951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:33.905951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:33.905951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:33.907673 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 306/614] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:42:33.911944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa1179-2fa1-4077-a0cc-d6f02300a0e2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:33.929591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-1be79b13-d38c-4538-a7ed-a3aeeb9b0803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:33.929591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:33.929591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:33.929591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:33.929591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:33.929591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:33.929591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:33.929591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:33.929591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:33.929591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:33.929591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:33.929591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:33.929591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:33.929591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:33.929591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:33.929591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:33.930593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:33.930593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:33.930593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:33.930593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:33.930593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:33.930593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:33.930593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:33.930593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:33.930593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:33.930593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:33.930593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:33.930593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:33.930593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:33.930593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:33.930593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:33.930593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:33.930593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:33.932575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:33.932575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:33.932575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:33.932575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 309/615] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:33.949176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be79b13-d38c-4538-a7ed-a3aeeb9b0803 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:34.013725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:34.031511 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa1179-2fa1-4077-a0cc-d6f02300a0e2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:34.059427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:34.076972 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314a2dc-2593-4648-baad-b5ebc5d0f55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:34.077503 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314a2dc-2593-4648-baad-b5ebc5d0f55b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:34.077853 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314a2dc-2593-4648-baad-b5ebc5d0f55b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:34.078127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314a2dc-2593-4648-baad-b5ebc5d0f55b None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:34.078807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314a2dc-2593-4648-baad-b5ebc5d0f55b None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:34.079111 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314a2dc-2593-4648-baad-b5ebc5d0f55b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:34.088731 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314a2dc-2593-4648-baad-b5ebc5d0f55b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:34.101114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 307/616] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:42:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:34.109049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314a2dc-2593-4648-baad-b5ebc5d0f55b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:34.137453 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be79b13-d38c-4538-a7ed-a3aeeb9b0803 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:34.209540 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddc7da-a0e5-4e92-9975-bfb1fcfc29b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:34.210324 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddc7da-a0e5-4e92-9975-bfb1fcfc29b8 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:34.210671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddc7da-a0e5-4e92-9975-bfb1fcfc29b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:34.210858 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddc7da-a0e5-4e92-9975-bfb1fcfc29b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:34.211519 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ddc7da-a0e5-4e92-9975-bfb1fcfc29b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:34.211820 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ddc7da-a0e5-4e92-9975-bfb1fcfc29b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:34.213244 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ddc7da-a0e5-4e92-9975-bfb1fcfc29b8 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:34.230067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 310/617] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:34.236939 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddc7da-a0e5-4e92-9975-bfb1fcfc29b8 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:34.339982 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314a2dc-2593-4648-baad-b5ebc5d0f55b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:34.379451 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18342d6-c574-4ca4-865a-0aaf14b8fb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:34.380123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18342d6-c574-4ca4-865a-0aaf14b8fb8d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:34.380409 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18342d6-c574-4ca4-865a-0aaf14b8fb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:34.380682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18342d6-c574-4ca4-865a-0aaf14b8fb8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:34.392658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18342d6-c574-4ca4-865a-0aaf14b8fb8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:34.394176 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18342d6-c574-4ca4-865a-0aaf14b8fb8d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:34.402276 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:34.411605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-c18342d6-c574-4ca4-865a-0aaf14b8fb8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:34.411605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:34.411605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:34.411605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:34.411605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:34.411605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:34.411605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:34.411605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:34.411605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:34.411605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:34.411605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:34.411605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:34.411605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:34.411605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:34.411605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:34.411605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:34.412702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:34.412702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:34.412702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:34.412702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:34.412702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:34.412702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:34.412702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:34.412702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:34.412702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:34.412702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:34.412702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:34.412702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:34.412702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:34.412702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:34.412702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:34.412702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:34.412702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:34.414055 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:34.414055 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:34.414055 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:34.414055 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 308/618] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:34.419505 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18342d6-c574-4ca4-865a-0aaf14b8fb8d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:34.466169 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddc7da-a0e5-4e92-9975-bfb1fcfc29b8 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:34.520515 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:34.549023 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338510e-a9ab-4cf8-9491-4bc06a4b9c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:34.549687 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338510e-a9ab-4cf8-9491-4bc06a4b9c7f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:34.549963 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338510e-a9ab-4cf8-9491-4bc06a4b9c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:34.550231 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338510e-a9ab-4cf8-9491-4bc06a4b9c7f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:34.551390 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18342d6-c574-4ca4-865a-0aaf14b8fb8d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:34.564187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338510e-a9ab-4cf8-9491-4bc06a4b9c7f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:34.566018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338510e-a9ab-4cf8-9491-4bc06a4b9c7f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:34.589583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-5338510e-a9ab-4cf8-9491-4bc06a4b9c7f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:42:34.589583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:34.589583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:34.589583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:34.589583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:34.589583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:34.589583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:34.589583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:34.589583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:34.589583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:34.589583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:34.589583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:34.589583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:34.589583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:34.589583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:34.589583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:34.590659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:34.590659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:34.590659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:34.590659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:34.590659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:34.590659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:34.590659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:34.590659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:34.590659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:34.590659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:34.590659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:34.590659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:34.590659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:34.590659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:34.590659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:34.590659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:34.590659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:34.590659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:34.591823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:34.591823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:34.591823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:34.591823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:34.591823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:34.591823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:34.591823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:34.591823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:34.591823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:42:34.591823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:34.596135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 311/619] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:42:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:34.599067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338510e-a9ab-4cf8-9491-4bc06a4b9c7f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:34.646427 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32faceb5-64c2-4b42-abfc-8061dff93327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:34.647605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32faceb5-64c2-4b42-abfc-8061dff93327 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:34.647873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32faceb5-64c2-4b42-abfc-8061dff93327 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:34.648192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32faceb5-64c2-4b42-abfc-8061dff93327 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:34.651182 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32faceb5-64c2-4b42-abfc-8061dff93327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:34.652377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32faceb5-64c2-4b42-abfc-8061dff93327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:34.653795 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32faceb5-64c2-4b42-abfc-8061dff93327 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:34.684541 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 309/620] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:34.690082 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32faceb5-64c2-4b42-abfc-8061dff93327 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:34.757280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:34.804939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32faceb5-64c2-4b42-abfc-8061dff93327 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:34.812307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338510e-a9ab-4cf8-9491-4bc06a4b9c7f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:34.851769 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b382a-0e2b-4311-80d2-012d6e29267c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:34.852787 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b382a-0e2b-4311-80d2-012d6e29267c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:34.853276 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b382a-0e2b-4311-80d2-012d6e29267c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:34.853775 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b382a-0e2b-4311-80d2-012d6e29267c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:34.854617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3610d8-b74a-4a51-9fde-576b530d0f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:34.855228 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3610d8-b74a-4a51-9fde-576b530d0f7b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:34.855504 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3610d8-b74a-4a51-9fde-576b530d0f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:34.855768 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3610d8-b74a-4a51-9fde-576b530d0f7b None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:34.856548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3610d8-b74a-4a51-9fde-576b530d0f7b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:34.856848 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3610d8-b74a-4a51-9fde-576b530d0f7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:34.858272 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3610d8-b74a-4a51-9fde-576b530d0f7b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:34.864923 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b382a-0e2b-4311-80d2-012d6e29267c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:34.867584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b382a-0e2b-4311-80d2-012d6e29267c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:34.878587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 312/621] 10.222.0.67 () {56 vars in 1135 bytes} [Wed Aug 24 08:42:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:34.883250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d91cd2-651a-4849-9af3-aa621bd64e49 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:34.883250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d91cd2-651a-4849-9af3-aa621bd64e49 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:34.883250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d91cd2-651a-4849-9af3-aa621bd64e49 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:34.883250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 313/622] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:34.888328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3d48f-9323-4356-97f6-7e02890f7421 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:34.888608 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3d48f-9323-4356-97f6-7e02890f7421 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:34.888880 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3d48f-9323-4356-97f6-7e02890f7421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:34.889443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-4b9b382a-0e2b-4311-80d2-012d6e29267c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:42:34.889443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:34.889443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:34.889443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:34.889443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:34.889443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:34.889443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:34.889443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:34.889443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:34.889443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:34.889443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:34.889443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:34.889443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:34.889443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:34.889443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:34.889443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:34.890356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:34.890356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:34.890356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:34.890356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:34.890356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:34.890356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:34.890356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:34.890356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:34.890356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:34.890356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:34.890356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:34.890356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:34.890356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:34.890356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:34.890356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:34.890356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:34.890356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:34.890356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:34.891320 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:34.891320 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:34.891320 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:34.891320 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:34.891320 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:34.891320 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:34.891320 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:34.891320 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:34.891320 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:42:34.891320 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:34.894576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 310/623] 10.222.0.67 () {56 vars in 1093 bytes} [Wed Aug 24 08:42:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:34.899225 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b382a-0e2b-4311-80d2-012d6e29267c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:34.922969 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-39d3d48f-9323-4356-97f6-7e02890f7421 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:34.925709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-39d3d48f-9323-4356-97f6-7e02890f7421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:34.975786 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:35.015824 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b382a-0e2b-4311-80d2-012d6e29267c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:35.023384 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 314/624] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:42:35.024574 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3d48f-9323-4356-97f6-7e02890f7421 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:35.055128 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:35.113610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a2a78-0197-4e11-b37c-b1f89caaa2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:35.114182 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a2a78-0197-4e11-b37c-b1f89caaa2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:35.114461 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a2a78-0197-4e11-b37c-b1f89caaa2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:35.114734 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a2a78-0197-4e11-b37c-b1f89caaa2c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:35.141134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a2a78-0197-4e11-b37c-b1f89caaa2c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:35.142626 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a2a78-0197-4e11-b37c-b1f89caaa2c0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:35.175823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-383a2a78-0197-4e11-b37c-b1f89caaa2c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:35.175823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:35.175823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:35.175823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:35.175823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:35.175823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:35.175823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:35.175823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:35.175823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:35.175823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:35.175823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:35.175823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:35.175823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:35.175823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:35.175823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:35.175823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:35.176915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:35.176915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:35.176915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:35.176915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:35.176915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:35.176915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:35.176915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:35.176915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:35.176915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:35.176915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:35.176915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:35.176915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:35.176915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:35.176915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:35.176915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:35.176915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:35.176915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:35.177817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:35.177817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:35.177817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:35.184582 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 311/625] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:35.185661 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3d48f-9323-4356-97f6-7e02890f7421 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:35.190177 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a2a78-0197-4e11-b37c-b1f89caaa2c0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:35.227403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9bace-6d49-4c6a-8584-d7a19405cb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:35.228534 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9bace-6d49-4c6a-8584-d7a19405cb90 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:35.229025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9bace-6d49-4c6a-8584-d7a19405cb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:35.229551 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9bace-6d49-4c6a-8584-d7a19405cb90 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:35.230504 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9bace-6d49-4c6a-8584-d7a19405cb90 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:35.231037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9bace-6d49-4c6a-8584-d7a19405cb90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:35.232771 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9bace-6d49-4c6a-8584-d7a19405cb90 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:35.255855 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 315/626] 10.222.0.67 () {56 vars in 1133 bytes} [Wed Aug 24 08:42:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:35.262129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9bace-6d49-4c6a-8584-d7a19405cb90 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:35.294987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:35.394409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9bace-6d49-4c6a-8584-d7a19405cb90 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:35.438479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a2a78-0197-4e11-b37c-b1f89caaa2c0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:35.448525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598194b-1c3e-4076-ad20-6a5b8a657f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:35.449301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598194b-1c3e-4076-ad20-6a5b8a657f26 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:35.449645 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598194b-1c3e-4076-ad20-6a5b8a657f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:35.449911 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598194b-1c3e-4076-ad20-6a5b8a657f26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:35.475950 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bbeb1-982f-4eee-b813-44c5f551a349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:35.476915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bbeb1-982f-4eee-b813-44c5f551a349 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:35.477278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bbeb1-982f-4eee-b813-44c5f551a349 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:35.477625 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bbeb1-982f-4eee-b813-44c5f551a349 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:35.478345 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bbeb1-982f-4eee-b813-44c5f551a349 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:35.478716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bbeb1-982f-4eee-b813-44c5f551a349 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:35.480440 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bbeb1-982f-4eee-b813-44c5f551a349 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:35.494239 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 312/627] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:35.495353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bbeb1-982f-4eee-b813-44c5f551a349 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:35.528571 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:35.536880 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598194b-1c3e-4076-ad20-6a5b8a657f26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:35.539250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598194b-1c3e-4076-ad20-6a5b8a657f26 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:35.572180 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:35.608082 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bbeb1-982f-4eee-b813-44c5f551a349 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:35.643803 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcc414-7e05-418d-b2d1-c6e48c8b198b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:35.644527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc414-7e05-418d-b2d1-c6e48c8b198b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:35.644746 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc414-7e05-418d-b2d1-c6e48c8b198b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:35.645016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc414-7e05-418d-b2d1-c6e48c8b198b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:35.655401 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcc414-7e05-418d-b2d1-c6e48c8b198b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:35.658986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 316/628] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 08:42:35.659683 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598194b-1c3e-4076-ad20-6a5b8a657f26 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:35.660068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-c6bcc414-7e05-418d-b2d1-c6e48c8b198b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:35.660068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:35.660068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:35.660068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:35.660068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:35.660068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:35.660068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:35.660068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:35.660068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:35.660068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:35.660068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:35.660068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:35.660068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:35.660068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:35.660068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:35.660068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:35.661001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:35.661001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:35.661001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:35.661001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:35.661001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:35.661001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:35.661001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:35.661001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:35.661001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:35.661001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:35.661001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:35.661001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:35.661001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:35.661001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:35.661001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:35.661001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:35.661926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:35.661926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:35.661926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:35.661926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:35.661926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:35.661926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:35.661926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:35.661926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:35.661926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:35.661926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:35.661926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:35.661926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:35.666504 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 313/629] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:35.667849 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ebcb2-3ff3-4435-a68d-7128fe8c2b42 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:35.668151 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ebcb2-3ff3-4435-a68d-7128fe8c2b42 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:35.668423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ebcb2-3ff3-4435-a68d-7128fe8c2b42 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:35.672141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 314/630] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:35.672141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ebcb2-3ff3-4435-a68d-7128fe8c2b42 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:35.778219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:35.785024 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598194b-1c3e-4076-ad20-6a5b8a657f26 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:35.792714 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ebcb2-3ff3-4435-a68d-7128fe8c2b42 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:35.819841 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:35.831405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866f130-be2a-4d16-ae17-53d0bc001025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:35.832875 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866f130-be2a-4d16-ae17-53d0bc001025 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:35.832875 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866f130-be2a-4d16-ae17-53d0bc001025 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:35.833643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866f130-be2a-4d16-ae17-53d0bc001025 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:35.838640 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce060f0-4244-4d45-8feb-28cb59fab29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:35.839625 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce060f0-4244-4d45-8feb-28cb59fab29e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:35.843609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce060f0-4244-4d45-8feb-28cb59fab29e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:35.843609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce060f0-4244-4d45-8feb-28cb59fab29e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:35.843976 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866f130-be2a-4d16-ae17-53d0bc001025 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:35.843976 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866f130-be2a-4d16-ae17-53d0bc001025 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:35.854618 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce060f0-4244-4d45-8feb-28cb59fab29e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:35.856071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce060f0-4244-4d45-8feb-28cb59fab29e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:35.856866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-d866f130-be2a-4d16-ae17-53d0bc001025 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:35.856866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:35.856866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:35.856866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:35.856866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:35.856866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:35.856866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:35.856866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:35.856866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:35.856866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:35.856866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:35.856866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:35.856866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:35.856866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:35.856866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:35.858328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:35.858328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:35.858328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:35.858328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:35.858328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:35.858328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:35.858328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:35.858328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:35.858328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:35.858328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:35.858328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:35.858328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:35.858328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:35.858328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:35.858328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:35.858328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:35.858328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:35.859402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:35.859402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:35.859402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:35.859402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:35.859402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:35.859402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:35.859402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:35.859402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:35.859402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 317/631] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:35.859402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866f130-be2a-4d16-ae17-53d0bc001025 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:35.876805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-bce060f0-4244-4d45-8feb-28cb59fab29e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:35.876805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:35.876805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:35.876805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:35.876805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:35.876805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:35.876805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:35.876805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:35.876805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:35.876805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:35.876805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:35.876805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:35.876805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:35.876805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:35.876805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:35.876805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:35.878519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:35.878519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:35.878519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:35.878519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:35.878519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:35.878519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:35.878519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:35.878519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:35.878519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:35.878519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:35.878519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:35.878519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:35.878519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:35.878519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:35.878519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:35.878519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:35.878519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:35.879862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:35.879862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:35.879862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:35.879862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 315/632] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:35.879862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5df9e0-670f-4dd2-80da-f77c18db1a31 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:35.879862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5df9e0-670f-4dd2-80da-f77c18db1a31 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:35.879862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5df9e0-670f-4dd2-80da-f77c18db1a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:35.897734 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7d5df9e0-670f-4dd2-80da-f77c18db1a31 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:35.898992 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7d5df9e0-670f-4dd2-80da-f77c18db1a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:35.968648 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866f130-be2a-4d16-ae17-53d0bc001025 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:35.969543 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 316/633] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:35.970815 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0711a10-65e9-4131-a594-89c3bc9e4e3b None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:35.971109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0711a10-65e9-4131-a594-89c3bc9e4e3b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:35.971370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0711a10-65e9-4131-a594-89c3bc9e4e3b None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:35.972661 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 317/634] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:35.973352 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0711a10-65e9-4131-a594-89c3bc9e4e3b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:36.020807 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e119e-ca04-4a0e-9532-641f0524cd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:36.021529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e119e-ca04-4a0e-9532-641f0524cd4f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:36.021815 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e119e-ca04-4a0e-9532-641f0524cd4f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:36.023046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e119e-ca04-4a0e-9532-641f0524cd4f None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:36.023748 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e119e-ca04-4a0e-9532-641f0524cd4f None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:36.024052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e119e-ca04-4a0e-9532-641f0524cd4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:36.025979 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e119e-ca04-4a0e-9532-641f0524cd4f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:36.041632 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 318/635] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:42:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:36.041632 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e119e-ca04-4a0e-9532-641f0524cd4f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:36.060759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:36.102200 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0711a10-65e9-4131-a594-89c3bc9e4e3b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:36.110419 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:36.153221 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2801ea-d9d7-4cf9-b72d-bcad9f3a8377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:36.154171 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2801ea-d9d7-4cf9-b72d-bcad9f3a8377 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:36.154675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2801ea-d9d7-4cf9-b72d-bcad9f3a8377 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:36.155156 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2801ea-d9d7-4cf9-b72d-bcad9f3a8377 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:36.157112 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2801ea-d9d7-4cf9-b72d-bcad9f3a8377 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:36.157112 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2801ea-d9d7-4cf9-b72d-bcad9f3a8377 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:36.159675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2801ea-d9d7-4cf9-b72d-bcad9f3a8377 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:36.172776 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e119e-ca04-4a0e-9532-641f0524cd4f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:36.186278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 318/636] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:36.186278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86411fb-2fa5-4441-bfc2-523b77d5d46a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:36.186278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86411fb-2fa5-4441-bfc2-523b77d5d46a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:36.186278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86411fb-2fa5-4441-bfc2-523b77d5d46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:36.201675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c86411fb-2fa5-4441-bfc2-523b77d5d46a None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:36.202880 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c86411fb-2fa5-4441-bfc2-523b77d5d46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:36.226617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ae031-d278-4245-9a49-3486e97c9030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:36.227238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ae031-d278-4245-9a49-3486e97c9030 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:36.228581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ae031-d278-4245-9a49-3486e97c9030 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:36.228581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ae031-d278-4245-9a49-3486e97c9030 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:36.228581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ae031-d278-4245-9a49-3486e97c9030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:36.228918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ae031-d278-4245-9a49-3486e97c9030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:36.230240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ae031-d278-4245-9a49-3486e97c9030 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:36.251716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 319/637] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:36.252874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ae031-d278-4245-9a49-3486e97c9030 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:36.301495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 319/638] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:36.302356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86411fb-2fa5-4441-bfc2-523b77d5d46a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:36.386657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ae031-d278-4245-9a49-3486e97c9030 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:36.389633 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:36.404043 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:36.437204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86411fb-2fa5-4441-bfc2-523b77d5d46a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:36.443333 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49ccf8-f08d-4b96-bd2a-9e5d609cb56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:36.444444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49ccf8-f08d-4b96-bd2a-9e5d609cb56a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:36.444889 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49ccf8-f08d-4b96-bd2a-9e5d609cb56a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:36.445329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49ccf8-f08d-4b96-bd2a-9e5d609cb56a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:36.458740 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49ccf8-f08d-4b96-bd2a-9e5d609cb56a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:36.460510 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49ccf8-f08d-4b96-bd2a-9e5d609cb56a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:36.474060 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b8169-6026-4040-aba1-d65bef3b8b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:36.475725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b8169-6026-4040-aba1-d65bef3b8b3a None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:36.475998 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b8169-6026-4040-aba1-d65bef3b8b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:36.476315 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b8169-6026-4040-aba1-d65bef3b8b3a None admin] PATH_INFO: `/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/46b1417d4ac24c8982b2052f4e99042a/roles/9aa748c8ea2542e6987e0b93190ea4e6` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:36.479062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-da49ccf8-f08d-4b96-bd2a-9e5d609cb56a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:36.479062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:36.479062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:36.479062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:36.479062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:36.479062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:36.479062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:36.479062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:36.479062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:36.479062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:36.479062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:36.479062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:36.479062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:36.479062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:36.479062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:36.479062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:36.480204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:36.480204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:36.480204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:36.480204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:36.480204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:36.480204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:36.480204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:36.480204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:36.480204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:36.480204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:36.480204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:36.480204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:36.480204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:36.480204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:36.480204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:36.480204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:36.480204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:36.485149 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:36.485149 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:36.485149 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:36.485149 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 320/639] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:36.485149 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49ccf8-f08d-4b96-bd2a-9e5d609cb56a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:36.503109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b8169-6026-4040-aba1-d65bef3b8b3a None admin] RBAC: Authorizing `identity:create_grant(project_id=fce6ee19f58b4a468c9a5d169e29a018, user_id=46b1417d4ac24c8982b2052f4e99042a, role_id=9aa748c8ea2542e6987e0b93190ea4e6)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:36.503862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b8169-6026-4040-aba1-d65bef3b8b3a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:36.519092 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 320/640] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:42:36 2022] PUT /identity/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/46b1417d4ac24c8982b2052f4e99042a/roles/9aa748c8ea2542e6987e0b93190ea4e6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:42:36.520523 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b8169-6026-4040-aba1-d65bef3b8b3a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:36.640434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49ccf8-f08d-4b96-bd2a-9e5d609cb56a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:36.685873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b8169-6026-4040-aba1-d65bef3b8b3a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:36.686066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c22a42-5b62-4fc9-94e9-babfb06137f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:36.686564 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c22a42-5b62-4fc9-94e9-babfb06137f9 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:36.686840 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c22a42-5b62-4fc9-94e9-babfb06137f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:36.687112 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c22a42-5b62-4fc9-94e9-babfb06137f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:36.695756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c22a42-5b62-4fc9-94e9-babfb06137f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:36.697282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c22a42-5b62-4fc9-94e9-babfb06137f9 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:36.709365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:36.713448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-63c22a42-5b62-4fc9-94e9-babfb06137f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:36.713448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:36.713448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:36.713448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:36.713448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:36.713448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:36.713448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:36.713448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:36.713448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:36.713448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:36.713448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:36.713448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:36.713448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:36.713448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:36.713448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:36.714619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:36.714619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:36.714619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:36.714619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:36.714619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:36.714619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:36.714619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:36.714619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:36.714619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:36.714619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:36.714619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:36.714619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:36.714619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:36.714619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:36.714619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:36.714619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:36.714619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:36.715669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:36.715669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:36.715669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:36.715669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:36.715669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:36.715669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:36.715669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:36.715669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:36.715669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 321/641] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:36.715669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c22a42-5b62-4fc9-94e9-babfb06137f9 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:36.754431 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:36.756399 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c5a45-18fc-45ee-810c-c1a8d03f345f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:36.757745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c5a45-18fc-45ee-810c-c1a8d03f345f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:36.758225 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c5a45-18fc-45ee-810c-c1a8d03f345f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:36.758694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c5a45-18fc-45ee-810c-c1a8d03f345f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:36.773108 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c5a45-18fc-45ee-810c-c1a8d03f345f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:36.780146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-287c5a45-18fc-45ee-810c-c1a8d03f345f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:36.780146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:36.780146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:36.780146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:36.780146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:36.780146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:36.780146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:36.780146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:36.780146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:36.780146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:36.780146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:36.780146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:36.780146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:36.780146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:36.780146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:36.780146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:36.781837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:36.781837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:36.781837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:36.781837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:36.781837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:36.781837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:36.781837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:36.781837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:36.781837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:36.781837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:36.781837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:36.781837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:36.781837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:36.781837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:36.781837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:36.781837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:36.782939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:36.782939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:36.782939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:36.782939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:36.782939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:36.782939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:36.782939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:36.782939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:36.782939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:36.782939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:36.782939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:36.782939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:36.784124 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 321/642] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:36.785442 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c5a45-18fc-45ee-810c-c1a8d03f345f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:36.879098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c22a42-5b62-4fc9-94e9-babfb06137f9 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:36.923963 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c5a45-18fc-45ee-810c-c1a8d03f345f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:36.924626 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15dccb2-f96d-461e-8c93-b5e21f0e4c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:36.925775 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dccb2-f96d-461e-8c93-b5e21f0e4c79 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:36.926051 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dccb2-f96d-461e-8c93-b5e21f0e4c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:36.926323 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dccb2-f96d-461e-8c93-b5e21f0e4c79 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:36.928885 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15dccb2-f96d-461e-8c93-b5e21f0e4c79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:36.929123 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15dccb2-f96d-461e-8c93-b5e21f0e4c79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:36.930515 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15dccb2-f96d-461e-8c93-b5e21f0e4c79 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:36.943798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 322/643] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:36.949139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15dccb2-f96d-461e-8c93-b5e21f0e4c79 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:36.965609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f718d-e340-4f91-96ba-0be985e6406c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:36.966244 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f718d-e340-4f91-96ba-0be985e6406c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:36.966536 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f718d-e340-4f91-96ba-0be985e6406c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:36.966803 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f718d-e340-4f91-96ba-0be985e6406c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:36.978090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f718d-e340-4f91-96ba-0be985e6406c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:36.982868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-6a9f718d-e340-4f91-96ba-0be985e6406c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:36.982868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:36.982868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:36.982868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:36.982868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:36.982868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:36.982868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:36.982868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:36.982868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:36.982868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:36.982868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:36.982868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:36.982868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:36.982868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:36.982868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:36.982868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:36.983796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:36.983796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:36.983796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:36.983796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:36.983796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:36.983796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:36.983796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:36.983796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:36.983796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:36.983796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:36.983796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:36.983796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:36.983796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:36.983796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:36.983796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:36.983796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:36.984726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:36.984726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:36.984726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:36.984726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:36.984726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:36.984726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:36.984726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:36.984726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:36.984726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:36.984726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:36.984726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:36.984726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:36.984726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 322/644] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:36.985523 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f718d-e340-4f91-96ba-0be985e6406c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:37.026241 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:37.055550 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:37.079488 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15dccb2-f96d-461e-8c93-b5e21f0e4c79 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:37.105541 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f718d-e340-4f91-96ba-0be985e6406c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:37.120900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c31ba5-b4ba-4e6b-aa4e-ed60afa621fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:37.121514 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c31ba5-b4ba-4e6b-aa4e-ed60afa621fb None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:37.121802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c31ba5-b4ba-4e6b-aa4e-ed60afa621fb None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:37.122060 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c31ba5-b4ba-4e6b-aa4e-ed60afa621fb None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:37.122753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c31ba5-b4ba-4e6b-aa4e-ed60afa621fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:37.123053 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c31ba5-b4ba-4e6b-aa4e-ed60afa621fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:37.124655 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c31ba5-b4ba-4e6b-aa4e-ed60afa621fb None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:37.142810 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 323/645] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:37.143711 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c31ba5-b4ba-4e6b-aa4e-ed60afa621fb None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:37.143889 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b3da4-0a26-4c0f-b85b-3d209de2f4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:37.144594 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b3da4-0a26-4c0f-b85b-3d209de2f4c2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:37.144874 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b3da4-0a26-4c0f-b85b-3d209de2f4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:37.145140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b3da4-0a26-4c0f-b85b-3d209de2f4c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:37.145806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b3da4-0a26-4c0f-b85b-3d209de2f4c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:37.146106 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b3da4-0a26-4c0f-b85b-3d209de2f4c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:37.148011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b3da4-0a26-4c0f-b85b-3d209de2f4c2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:37.158383 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 323/646] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:42:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:37.159069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b3da4-0a26-4c0f-b85b-3d209de2f4c2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:37.271351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c31ba5-b4ba-4e6b-aa4e-ed60afa621fb None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:37.285173 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:37.293862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b3da4-0a26-4c0f-b85b-3d209de2f4c2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:37.316802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89cbf3-f5f8-40f8-bdd1-27c9273ee425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:37.317593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89cbf3-f5f8-40f8-bdd1-27c9273ee425 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:37.317971 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89cbf3-f5f8-40f8-bdd1-27c9273ee425 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:37.318396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89cbf3-f5f8-40f8-bdd1-27c9273ee425 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:37.328147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89cbf3-f5f8-40f8-bdd1-27c9273ee425 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:37.329858 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89cbf3-f5f8-40f8-bdd1-27c9273ee425 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:37.332667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ae36a-52e2-4ac1-8169-1597be323fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:37.333433 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ae36a-52e2-4ac1-8169-1597be323fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:37.333810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ae36a-52e2-4ac1-8169-1597be323fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:37.334183 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ae36a-52e2-4ac1-8169-1597be323fb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:37.335589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ae36a-52e2-4ac1-8169-1597be323fb6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:37.336041 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ae36a-52e2-4ac1-8169-1597be323fb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:37.337610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ae36a-52e2-4ac1-8169-1597be323fb6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:37.343147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-fb89cbf3-f5f8-40f8-bdd1-27c9273ee425 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 08:42:37.343147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:37.343147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:37.343147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:37.343147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:37.343147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:37.343147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:37.343147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:37.343147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:37.343147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:37.343147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:37.343147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:37.343147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:37.343147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:37.343147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:37.343147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:37.344389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:37.344389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:37.344389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:37.344389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:37.344389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:37.344389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:37.344389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:37.344389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:37.344389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:37.344389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:37.344389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:37.344389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:37.344389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:37.344389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:37.344389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:37.344389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:37.344389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:37.344389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:37.345332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:37.345332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:37.345332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:37.345332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:37.345332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:37.345332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:37.345332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:37.345332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:37.345332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 08:42:37.345332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:37.345332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 324/647] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:42:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:37.345925 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89cbf3-f5f8-40f8-bdd1-27c9273ee425 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:37.348717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:37.354389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 324/648] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:42:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:37.355630 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ae36a-52e2-4ac1-8169-1597be323fb6 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:37.476205 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ae36a-52e2-4ac1-8169-1597be323fb6 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:37.483092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89cbf3-f5f8-40f8-bdd1-27c9273ee425 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:37.515784 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb0b24-0f43-4100-bc40-16fd15d14ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:37.516549 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb0b24-0f43-4100-bc40-16fd15d14ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:37.516825 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb0b24-0f43-4100-bc40-16fd15d14ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:37.517107 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb0b24-0f43-4100-bc40-16fd15d14ba2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:37.522144 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a601d-d71d-4194-bbbf-7d2f0b75b2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:37.523036 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a601d-d71d-4194-bbbf-7d2f0b75b2ef None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:37.523419 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a601d-d71d-4194-bbbf-7d2f0b75b2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:37.523814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a601d-d71d-4194-bbbf-7d2f0b75b2ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:37.524677 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a601d-d71d-4194-bbbf-7d2f0b75b2ef None admin] RBAC: Adding query filter params (role.id=9aa748c8ea2542e6987e0b93190ea4e6, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=ca56dd7216fb4946946b2801e144fec2) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:37.525086 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a601d-d71d-4194-bbbf-7d2f0b75b2ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:37.526627 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a601d-d71d-4194-bbbf-7d2f0b75b2ef None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:37.529852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb0b24-0f43-4100-bc40-16fd15d14ba2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:37.531300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb0b24-0f43-4100-bc40-16fd15d14ba2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:37.534124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 325/649] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:42:37 2022] GET /identity/v3/role_assignments?user.id=ca56dd7216fb4946946b2801e144fec2&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=9aa748c8ea2542e6987e0b93190ea4e6 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:37.534794 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a601d-d71d-4194-bbbf-7d2f0b75b2ef None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:37.564411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-5dcb0b24-0f43-4100-bc40-16fd15d14ba2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:37.564411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:37.564411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:37.564411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:37.564411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:37.564411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:37.564411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:37.564411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:37.564411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:37.564411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:37.564411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:37.564411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:37.564411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:37.564411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:37.564411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:37.564411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:37.565647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:37.565647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:37.565647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:37.565647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:37.565647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:37.565647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:37.565647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:37.565647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:37.565647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:37.565647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:37.565647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:37.565647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:37.565647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:37.565647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:37.565647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:37.565647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:37.565647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:37.566745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:37.566745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:37.566745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:37.574166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 325/650] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:37.574166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb0b24-0f43-4100-bc40-16fd15d14ba2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:37.591737 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:37.630984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:37.685033 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a601d-d71d-4194-bbbf-7d2f0b75b2ef None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:37.711817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb0b24-0f43-4100-bc40-16fd15d14ba2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:37.734413 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06d268-33c5-4d83-ae9b-902db9d47b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:37.735063 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06d268-33c5-4d83-ae9b-902db9d47b86 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:37.735343 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06d268-33c5-4d83-ae9b-902db9d47b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:37.735627 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06d268-33c5-4d83-ae9b-902db9d47b86 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:37.736541 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a06d268-33c5-4d83-ae9b-902db9d47b86 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:37.736852 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a06d268-33c5-4d83-ae9b-902db9d47b86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:37.739180 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a06d268-33c5-4d83-ae9b-902db9d47b86 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:37.761787 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 326/651] 10.222.0.67 () {56 vars in 1137 bytes} [Wed Aug 24 08:42:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:37.762691 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06d268-33c5-4d83-ae9b-902db9d47b86 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:37.770256 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d164f8f-e170-4c2a-b0c3-b276ffed8578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:37.770983 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d164f8f-e170-4c2a-b0c3-b276ffed8578 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:37.771338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d164f8f-e170-4c2a-b0c3-b276ffed8578 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:37.771711 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d164f8f-e170-4c2a-b0c3-b276ffed8578 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:37.786796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d164f8f-e170-4c2a-b0c3-b276ffed8578 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:37.788510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d164f8f-e170-4c2a-b0c3-b276ffed8578 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:37.806788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-8d164f8f-e170-4c2a-b0c3-b276ffed8578 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:37.806788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:37.806788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:37.806788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:37.806788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:37.806788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:37.806788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:37.806788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:37.806788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:37.806788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:37.806788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:37.806788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:37.806788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:37.806788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:37.806788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:37.806788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:37.807792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:37.807792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:37.807792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:37.807792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:37.807792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:37.807792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:37.807792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:37.807792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:37.807792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:37.807792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:37.807792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:37.807792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:37.807792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:37.807792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:37.807792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:37.807792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:37.807792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:37.809262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:37.809262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:37.809262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:37.809262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 326/652] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:37.809680 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d164f8f-e170-4c2a-b0c3-b276ffed8578 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:37.891789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:37.917312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06d268-33c5-4d83-ae9b-902db9d47b86 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:37.946748 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d164f8f-e170-4c2a-b0c3-b276ffed8578 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:37.952590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:37.969065 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7aa27b-b269-4683-8a06-464b281df348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:37.970274 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7aa27b-b269-4683-8a06-464b281df348 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:37.971174 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7aa27b-b269-4683-8a06-464b281df348 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:37.971621 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7aa27b-b269-4683-8a06-464b281df348 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:37.972485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7aa27b-b269-4683-8a06-464b281df348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:37.972962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7aa27b-b269-4683-8a06-464b281df348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:37.975058 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7aa27b-b269-4683-8a06-464b281df348 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:37.992870 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f5288-61e0-4cda-85bb-673dfc893c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:37.993597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f5288-61e0-4cda-85bb-673dfc893c31 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:37.994005 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f5288-61e0-4cda-85bb-673dfc893c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:37.994402 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f5288-61e0-4cda-85bb-673dfc893c31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:37.994699 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 327/653] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:37.995889 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7aa27b-b269-4683-8a06-464b281df348 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:38.078479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f5288-61e0-4cda-85bb-673dfc893c31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:38.080333 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f5288-61e0-4cda-85bb-673dfc893c31 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:38.124216 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7aa27b-b269-4683-8a06-464b281df348 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:38.147966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:38.165010 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b2380-f9d0-48a5-8b15-72950bfe0327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:38.165632 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b2380-f9d0-48a5-8b15-72950bfe0327 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:38.165912 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b2380-f9d0-48a5-8b15-72950bfe0327 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:38.166185 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b2380-f9d0-48a5-8b15-72950bfe0327 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:38.166828 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b2380-f9d0-48a5-8b15-72950bfe0327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:38.167130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b2380-f9d0-48a5-8b15-72950bfe0327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:38.169033 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b2380-f9d0-48a5-8b15-72950bfe0327 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:38.187870 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 328/654] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:38.188903 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b2380-f9d0-48a5-8b15-72950bfe0327 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:38.199975 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 327/655] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:42:38.201340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f5288-61e0-4cda-85bb-673dfc893c31 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:38.239132 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:38.301639 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b2380-f9d0-48a5-8b15-72950bfe0327 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:38.310586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f5288-61e0-4cda-85bb-673dfc893c31 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:38.340605 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86692d65-368d-4708-9c60-ea0addea5807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:38.341298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86692d65-368d-4708-9c60-ea0addea5807 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:38.341632 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86692d65-368d-4708-9c60-ea0addea5807 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:38.341906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86692d65-368d-4708-9c60-ea0addea5807 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:38.349229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f2de1-cf0e-4354-8293-1191ef1f57e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:38.350238 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f2de1-cf0e-4354-8293-1191ef1f57e4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:38.350540 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86692d65-368d-4708-9c60-ea0addea5807 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:38.351143 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f2de1-cf0e-4354-8293-1191ef1f57e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:38.351631 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f2de1-cf0e-4354-8293-1191ef1f57e4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:38.351997 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86692d65-368d-4708-9c60-ea0addea5807 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:38.363377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f2de1-cf0e-4354-8293-1191ef1f57e4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:38.365422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f2de1-cf0e-4354-8293-1191ef1f57e4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:38.367049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-86692d65-368d-4708-9c60-ea0addea5807 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:42:38.367049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:38.367049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:38.367049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:38.367049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:38.367049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:38.367049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:38.367049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:38.367049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:38.367049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:38.367049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:38.367049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:38.367049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:38.367049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:38.367049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:38.367049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:38.368465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:38.368465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:38.368465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:38.368465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:38.368465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:38.368465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:38.368465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:38.368465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:38.368465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:38.368465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:38.368465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:38.368465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:38.368465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:38.368465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:38.368465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:38.368465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:38.368465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:38.368465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:38.369957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:38.369957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:38.369957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:38.369957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:38.369957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:38.369957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:38.369957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:38.369957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:38.369957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:42:38.369957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:38.369957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 329/656] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:42:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:38.369957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86692d65-368d-4708-9c60-ea0addea5807 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:38.392854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-499f2de1-cf0e-4354-8293-1191ef1f57e4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 08:42:38.392854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:38.392854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:38.392854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:38.392854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:38.392854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:38.392854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:38.392854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:38.392854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:38.392854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:38.392854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:38.392854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:38.392854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:38.392854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:38.392854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:38.392854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:38.394693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:38.394693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:38.394693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:38.394693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:38.394693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:38.394693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:38.394693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:38.394693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:38.394693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:38.394693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:38.394693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:38.394693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:38.394693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:38.394693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:38.394693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:38.394693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:38.394693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:38.394693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:38.396604 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:38.396604 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:38.396604 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:38.396604 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:38.396604 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:38.396604 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:38.396604 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:38.396604 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:38.396604 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 08:42:38.396604 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:38.401596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 328/657] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:38.403454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f2de1-cf0e-4354-8293-1191ef1f57e4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:38.452468 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:38.490447 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86692d65-368d-4708-9c60-ea0addea5807 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:38.494309 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:38.523530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f2de1-cf0e-4354-8293-1191ef1f57e4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:38.530400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bf844-6956-4753-abe1-d53e8711a271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:38.531190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bf844-6956-4753-abe1-d53e8711a271 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:38.531575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bf844-6956-4753-abe1-d53e8711a271 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:38.531947 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bf844-6956-4753-abe1-d53e8711a271 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:38.543650 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3bf844-6956-4753-abe1-d53e8711a271 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:38.545326 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3bf844-6956-4753-abe1-d53e8711a271 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:38.561802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-fd3bf844-6956-4753-abe1-d53e8711a271 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:38.561802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:38.561802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:38.561802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:38.561802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:38.561802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:38.561802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:38.561802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:38.561802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:38.561802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:38.561802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:38.561802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:38.561802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:38.561802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:38.561802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:38.561802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:38.562895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:38.562895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:38.562895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:38.562895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:38.562895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:38.562895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:38.562895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:38.562895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:38.562895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:38.562895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:38.562895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:38.562895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:38.562895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:38.562895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:38.562895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:38.562895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:38.562895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:38.563789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:38.563789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:38.563789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:38.563789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 330/658] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:38.564887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bf844-6956-4753-abe1-d53e8711a271 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:38.566624 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05a50e-d2c2-441c-a701-0eba5fec32e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:38.567468 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05a50e-d2c2-441c-a701-0eba5fec32e6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:38.567950 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05a50e-d2c2-441c-a701-0eba5fec32e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:38.568495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05a50e-d2c2-441c-a701-0eba5fec32e6 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:38.570211 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05a50e-d2c2-441c-a701-0eba5fec32e6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:38.570759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05a50e-d2c2-441c-a701-0eba5fec32e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:38.572552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05a50e-d2c2-441c-a701-0eba5fec32e6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:38.594781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 329/659] 10.222.0.67 () {56 vars in 1137 bytes} [Wed Aug 24 08:42:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:38.595947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05a50e-d2c2-441c-a701-0eba5fec32e6 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:38.684988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bf844-6956-4753-abe1-d53e8711a271 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:38.695806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:38.714703 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05a50e-d2c2-441c-a701-0eba5fec32e6 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:38.726070 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2cdda-d2dc-4e99-ae95-934bc55e4ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:38.726695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2cdda-d2dc-4e99-ae95-934bc55e4ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:38.726973 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2cdda-d2dc-4e99-ae95-934bc55e4ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:38.727247 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2cdda-d2dc-4e99-ae95-934bc55e4ff1 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:38.727984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2cdda-d2dc-4e99-ae95-934bc55e4ff1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:38.728319 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2cdda-d2dc-4e99-ae95-934bc55e4ff1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:38.729756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2cdda-d2dc-4e99-ae95-934bc55e4ff1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:38.758083 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 331/660] 10.222.0.67 () {56 vars in 1139 bytes} [Wed Aug 24 08:42:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:38.758083 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2cdda-d2dc-4e99-ae95-934bc55e4ff1 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:38.768739 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51285fb6-3ed3-4783-bafc-6400ebb00b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:38.769707 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51285fb6-3ed3-4783-bafc-6400ebb00b1c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:38.770160 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51285fb6-3ed3-4783-bafc-6400ebb00b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:38.770605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51285fb6-3ed3-4783-bafc-6400ebb00b1c None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:38.771687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51285fb6-3ed3-4783-bafc-6400ebb00b1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:38.772245 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51285fb6-3ed3-4783-bafc-6400ebb00b1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:38.775657 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51285fb6-3ed3-4783-bafc-6400ebb00b1c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:38.783499 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:38.804918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 330/661] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:38.804918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67546489-4f33-4595-84a7-13e9520715c1 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:38.804918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67546489-4f33-4595-84a7-13e9520715c1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:38.804918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67546489-4f33-4595-84a7-13e9520715c1 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:38.804918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 331/662] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:38.805737 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67546489-4f33-4595-84a7-13e9520715c1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:38.910510 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2cdda-d2dc-4e99-ae95-934bc55e4ff1 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:38.952548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e570d-eb1d-4c88-92bb-516c1fa67edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:38.953174 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e570d-eb1d-4c88-92bb-516c1fa67edf None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:38.953648 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e570d-eb1d-4c88-92bb-516c1fa67edf None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:38.953757 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e570d-eb1d-4c88-92bb-516c1fa67edf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:38.962068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67546489-4f33-4595-84a7-13e9520715c1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:38.998629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0e6e2-554e-4a51-b2e8-2418e1f0cc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:38.999253 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0e6e2-554e-4a51-b2e8-2418e1f0cc0f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:38.999589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0e6e2-554e-4a51-b2e8-2418e1f0cc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:38.999862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0e6e2-554e-4a51-b2e8-2418e1f0cc0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:39.009542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:39.034847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e570d-eb1d-4c88-92bb-516c1fa67edf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:39.038111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e570d-eb1d-4c88-92bb-516c1fa67edf None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:39.066566 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:39.075738 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0e6e2-554e-4a51-b2e8-2418e1f0cc0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:39.077267 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0e6e2-554e-4a51-b2e8-2418e1f0cc0f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:39.182818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 332/663] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:42:39.182818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e570d-eb1d-4c88-92bb-516c1fa67edf None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:39.205173 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 332/664] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:42:39.205173 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91335771-f8ff-443d-91bd-fae3732ff740 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:39.205173 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91335771-f8ff-443d-91bd-fae3732ff740 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:39.205173 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91335771-f8ff-443d-91bd-fae3732ff740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:39.226921 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-91335771-f8ff-443d-91bd-fae3732ff740 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:39.228294 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-91335771-f8ff-443d-91bd-fae3732ff740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:39.267432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:39.291617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e570d-eb1d-4c88-92bb-516c1fa67edf None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:39.316249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:39.317664 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 333/665] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:39.318715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91335771-f8ff-443d-91bd-fae3732ff740 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:39.347457 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea1cb5-29b8-4dbc-8ce4-1c7b0f6f4a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:39.348313 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea1cb5-29b8-4dbc-8ce4-1c7b0f6f4a58 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:39.348712 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea1cb5-29b8-4dbc-8ce4-1c7b0f6f4a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:39.349076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea1cb5-29b8-4dbc-8ce4-1c7b0f6f4a58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:39.365402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea1cb5-29b8-4dbc-8ce4-1c7b0f6f4a58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:39.370330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea1cb5-29b8-4dbc-8ce4-1c7b0f6f4a58 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:39.399587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-6cea1cb5-29b8-4dbc-8ce4-1c7b0f6f4a58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:39.399587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:39.399587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:39.399587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:39.399587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:39.399587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:39.399587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:39.399587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:39.399587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:39.399587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:39.399587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:39.399587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:39.399587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:39.399587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:39.399587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:39.400638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:39.400638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:39.400638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:39.400638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:39.400638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:39.400638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:39.400638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:39.400638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:39.400638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:39.400638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:39.400638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:39.400638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:39.400638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:39.400638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:39.400638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:39.400638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:39.400638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:39.401553 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:39.401553 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:39.401553 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:39.401553 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:39.401553 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:39.401553 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:39.401553 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:39.401553 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:39.404294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 333/666] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:39.405506 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea1cb5-29b8-4dbc-8ce4-1c7b0f6f4a58 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:39.486435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91335771-f8ff-443d-91bd-fae3732ff740 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:39.528031 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b80d0b-eb6a-43bc-a1b4-49978f0be652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:39.528718 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b80d0b-eb6a-43bc-a1b4-49978f0be652 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:39.529015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b80d0b-eb6a-43bc-a1b4-49978f0be652 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:39.529340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b80d0b-eb6a-43bc-a1b4-49978f0be652 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:39.541773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b80d0b-eb6a-43bc-a1b4-49978f0be652 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:39.543257 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b80d0b-eb6a-43bc-a1b4-49978f0be652 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:39.564533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-c6b80d0b-eb6a-43bc-a1b4-49978f0be652 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:39.564533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:39.564533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:39.564533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:39.564533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:39.564533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:39.564533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:39.564533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:39.564533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:39.564533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:39.564533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:39.564533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:39.564533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:39.564533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:39.564533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:39.564533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:39.565558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:39.565558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:39.565558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:39.565558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:39.565558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:39.565558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:39.565558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:39.565558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:39.565558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:39.565558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:39.565558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:39.565558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:39.565558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:39.565558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:39.565558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:39.565558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:39.565558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:39.566460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:39.566460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:39.566460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:39.566460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 334/667] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:39.567119 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b80d0b-eb6a-43bc-a1b4-49978f0be652 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:39.575077 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea1cb5-29b8-4dbc-8ce4-1c7b0f6f4a58 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:39.601669 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:39.618220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598df653-8d69-49a9-b69b-90e5180cd982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:39.619044 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598df653-8d69-49a9-b69b-90e5180cd982 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:39.619422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598df653-8d69-49a9-b69b-90e5180cd982 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:39.619801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598df653-8d69-49a9-b69b-90e5180cd982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:39.631106 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598df653-8d69-49a9-b69b-90e5180cd982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:39.632986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598df653-8d69-49a9-b69b-90e5180cd982 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:39.651528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-598df653-8d69-49a9-b69b-90e5180cd982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:39.651528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:39.651528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:39.651528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:39.651528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:39.651528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:39.651528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:39.651528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:39.651528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:39.651528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:39.651528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:39.651528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:39.651528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:39.651528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:39.651528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:39.651528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:39.653190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:39.653190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:39.653190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:39.653190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:39.653190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:39.653190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:39.653190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:39.653190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:39.653190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:39.653190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:39.653190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:39.653190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:39.653190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:39.653190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:39.653190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:39.653190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:39.653190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:39.654552 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:39.654552 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:39.654552 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:39.655663 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 334/668] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:39.656223 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598df653-8d69-49a9-b69b-90e5180cd982 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:39.690695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:39.704409 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b80d0b-eb6a-43bc-a1b4-49978f0be652 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:39.749032 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc18081a-d24d-491d-b0cc-966f4bd978ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:39.749667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18081a-d24d-491d-b0cc-966f4bd978ae None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:39.749945 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18081a-d24d-491d-b0cc-966f4bd978ae None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:39.750220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18081a-d24d-491d-b0cc-966f4bd978ae None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:39.750937 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc18081a-d24d-491d-b0cc-966f4bd978ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:39.751253 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc18081a-d24d-491d-b0cc-966f4bd978ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:39.752809 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc18081a-d24d-491d-b0cc-966f4bd978ae None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:39.768116 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 335/669] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:39.769005 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc18081a-d24d-491d-b0cc-966f4bd978ae None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:39.783004 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598df653-8d69-49a9-b69b-90e5180cd982 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:39.821145 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457c0d7-4cc1-4ee4-ac92-82ce7899ed75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:39.821577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457c0d7-4cc1-4ee4-ac92-82ce7899ed75 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:39.821853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457c0d7-4cc1-4ee4-ac92-82ce7899ed75 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:39.822120 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457c0d7-4cc1-4ee4-ac92-82ce7899ed75 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:39.822789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457c0d7-4cc1-4ee4-ac92-82ce7899ed75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:39.823169 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457c0d7-4cc1-4ee4-ac92-82ce7899ed75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:39.824728 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457c0d7-4cc1-4ee4-ac92-82ce7899ed75 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:39.840173 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 335/670] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:39.841049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457c0d7-4cc1-4ee4-ac92-82ce7899ed75 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:39.868793 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:39.890631 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc18081a-d24d-491d-b0cc-966f4bd978ae None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:39.930048 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e93b46-8910-472b-99cd-dbfac4af6b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:39.930675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e93b46-8910-472b-99cd-dbfac4af6b20 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:39.930953 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e93b46-8910-472b-99cd-dbfac4af6b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:39.931223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e93b46-8910-472b-99cd-dbfac4af6b20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:39.941877 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e93b46-8910-472b-99cd-dbfac4af6b20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:39.942062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:39.947337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d0e93b46-8910-472b-99cd-dbfac4af6b20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:39.947337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:39.947337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:39.947337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:39.947337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:39.947337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:39.947337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:39.947337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:39.947337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:39.947337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:39.947337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:39.947337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:39.947337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:39.947337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:39.947337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:39.947337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:39.948340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:39.948340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:39.948340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:39.948340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:39.948340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:39.948340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:39.948340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:39.948340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:39.948340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:39.948340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:39.948340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:39.948340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:39.948340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:39.948340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:39.948340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:39.948340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:39.949220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:39.949220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:39.949220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:39.949220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:39.949220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:39.949220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:39.949220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:39.949220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:39.949220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:39.949220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:39.949220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:39.949220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:39.949220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 336/671] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:39.949953 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9dc35-2d9c-41a3-a342-febf6d35b9c9 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:39.949953 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9dc35-2d9c-41a3-a342-febf6d35b9c9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:39.950402 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9dc35-2d9c-41a3-a342-febf6d35b9c9 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:39.951409 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 337/672] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:39.952113 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9dc35-2d9c-41a3-a342-febf6d35b9c9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:39.964138 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457c0d7-4cc1-4ee4-ac92-82ce7899ed75 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:40.009414 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08bb9d-6cd7-4eff-972d-c9935b8b61c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:40.009953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08bb9d-6cd7-4eff-972d-c9935b8b61c1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:40.010233 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08bb9d-6cd7-4eff-972d-c9935b8b61c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:40.010598 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08bb9d-6cd7-4eff-972d-c9935b8b61c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:40.011347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08bb9d-6cd7-4eff-972d-c9935b8b61c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:40.011653 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08bb9d-6cd7-4eff-972d-c9935b8b61c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:40.013150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08bb9d-6cd7-4eff-972d-c9935b8b61c1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:40.025748 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 336/673] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:40.026602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08bb9d-6cd7-4eff-972d-c9935b8b61c1 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:40.070589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9dc35-2d9c-41a3-a342-febf6d35b9c9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:40.107601 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93db40a-5c28-4344-b049-da8ecd7279a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:40.108244 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93db40a-5c28-4344-b049-da8ecd7279a0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:40.108522 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93db40a-5c28-4344-b049-da8ecd7279a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:40.108793 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93db40a-5c28-4344-b049-da8ecd7279a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:40.109518 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93db40a-5c28-4344-b049-da8ecd7279a0 None admin] RBAC: Adding query filter params (role.id=9aa748c8ea2542e6987e0b93190ea4e6, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=5979387f7aa54619b8d429368c306cc9) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:40.109823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93db40a-5c28-4344-b049-da8ecd7279a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:40.111266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93db40a-5c28-4344-b049-da8ecd7279a0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:40.124166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 338/674] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:42:39 2022] GET /identity/v3/role_assignments?user.id=5979387f7aa54619b8d429368c306cc9&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=9aa748c8ea2542e6987e0b93190ea4e6 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:40.124166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93db40a-5c28-4344-b049-da8ecd7279a0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:40.127418 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:40.147996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08bb9d-6cd7-4eff-972d-c9935b8b61c1 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:40.192201 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b15cb-7eff-44c8-bd50-2c7123ce5609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:40.192960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b15cb-7eff-44c8-bd50-2c7123ce5609 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:40.193268 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b15cb-7eff-44c8-bd50-2c7123ce5609 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:40.193551 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b15cb-7eff-44c8-bd50-2c7123ce5609 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:40.202219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b15cb-7eff-44c8-bd50-2c7123ce5609 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:40.203721 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b15cb-7eff-44c8-bd50-2c7123ce5609 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:40.217133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-7d5b15cb-7eff-44c8-bd50-2c7123ce5609 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:40.217133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:40.217133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:40.217133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:40.217133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:40.217133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:40.217133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:40.217133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:40.217133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:40.217133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:40.217133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:40.217133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:40.217133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:40.217133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:40.217133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:40.218987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:40.218987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:40.218987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:40.218987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:40.218987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:40.218987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:40.218987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:40.218987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:40.218987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:40.218987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:40.218987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:40.218987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:40.218987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:40.218987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:40.218987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:40.218987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:40.218987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:40.219906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:40.219906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:40.219906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:40.219906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:40.219906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:40.219906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:40.219906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:40.219906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:40.223128 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 337/675] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:40.224582 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91748e-1a6b-41ee-968b-185bf98ce804 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:40.224862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91748e-1a6b-41ee-968b-185bf98ce804 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:40.225130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91748e-1a6b-41ee-968b-185bf98ce804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:40.238716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:40.247962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-fc91748e-1a6b-41ee-968b-185bf98ce804 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:40.249328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fc91748e-1a6b-41ee-968b-185bf98ce804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:40.254064 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93db40a-5c28-4344-b049-da8ecd7279a0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:40.299450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5884252-557b-4c47-b238-5d0d8f761e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:40.301285 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5884252-557b-4c47-b238-5d0d8f761e82 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:40.301633 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5884252-557b-4c47-b238-5d0d8f761e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:40.302557 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5884252-557b-4c47-b238-5d0d8f761e82 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:40.302704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5884252-557b-4c47-b238-5d0d8f761e82 None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:40.303073 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5884252-557b-4c47-b238-5d0d8f761e82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:40.306440 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5884252-557b-4c47-b238-5d0d8f761e82 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:40.316187 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 339/676] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:42:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:40.317118 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5884252-557b-4c47-b238-5d0d8f761e82 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:40.361103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 338/677] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:40.362071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91748e-1a6b-41ee-968b-185bf98ce804 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:40.416152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:40.470233 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5884252-557b-4c47-b238-5d0d8f761e82 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:40.503362 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91748e-1a6b-41ee-968b-185bf98ce804 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:40.511054 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8fb3a-9e87-4c81-9730-98a83255194e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:40.511976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8fb3a-9e87-4c81-9730-98a83255194e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:40.512580 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8fb3a-9e87-4c81-9730-98a83255194e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:40.513063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8fb3a-9e87-4c81-9730-98a83255194e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:40.521205 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8fb3a-9e87-4c81-9730-98a83255194e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:40.524174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8fb3a-9e87-4c81-9730-98a83255194e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:40.537192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-05e8fb3a-9e87-4c81-9730-98a83255194e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:40.537192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:40.537192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:40.537192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:40.537192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:40.537192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:40.537192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:40.537192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:40.537192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:40.537192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:40.537192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:40.537192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:40.537192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:40.537192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:40.537192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:40.538419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:40.538419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:40.538419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:40.538419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:40.538419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:40.538419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:40.538419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:40.538419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:40.538419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:40.538419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:40.538419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:40.538419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:40.538419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:40.538419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:40.538419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:40.538419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:40.538419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:40.539646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:40.539646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:40.539646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:40.539646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:40.539646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:40.539646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:40.539646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:40.539646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:40.542317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 340/678] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:40.543796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8fb3a-9e87-4c81-9730-98a83255194e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:40.544698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a057cf2-6c14-4f95-859c-4ebd519b91f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:40.545415 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a057cf2-6c14-4f95-859c-4ebd519b91f2 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:40.545766 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a057cf2-6c14-4f95-859c-4ebd519b91f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:40.546107 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a057cf2-6c14-4f95-859c-4ebd519b91f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:40.546884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a057cf2-6c14-4f95-859c-4ebd519b91f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:40.547271 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a057cf2-6c14-4f95-859c-4ebd519b91f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:40.549204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a057cf2-6c14-4f95-859c-4ebd519b91f2 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:40.554111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:40.573107 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 339/679] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:40.573107 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a057cf2-6c14-4f95-859c-4ebd519b91f2 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:40.683915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8fb3a-9e87-4c81-9730-98a83255194e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:40.700599 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a057cf2-6c14-4f95-859c-4ebd519b91f2 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:40.722240 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:40.730644 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfbb08-c4d7-4d84-9322-d53dd89fd678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:40.731277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfbb08-c4d7-4d84-9322-d53dd89fd678 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:40.731550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfbb08-c4d7-4d84-9322-d53dd89fd678 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:40.731822 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfbb08-c4d7-4d84-9322-d53dd89fd678 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:40.744289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfbb08-c4d7-4d84-9322-d53dd89fd678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:40.745245 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020942a3-c5bf-463b-9102-7273c1ff4f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:40.745748 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfbb08-c4d7-4d84-9322-d53dd89fd678 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:40.745857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020942a3-c5bf-463b-9102-7273c1ff4f0d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:40.746169 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020942a3-c5bf-463b-9102-7273c1ff4f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:40.746407 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020942a3-c5bf-463b-9102-7273c1ff4f0d None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:40.747124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020942a3-c5bf-463b-9102-7273c1ff4f0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:40.747428 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020942a3-c5bf-463b-9102-7273c1ff4f0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:40.748930 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020942a3-c5bf-463b-9102-7273c1ff4f0d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:40.762456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-25bfbb08-c4d7-4d84-9322-d53dd89fd678 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:40.762456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:40.762456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:40.762456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:40.762456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:40.762456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:40.762456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:40.762456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:40.762456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:40.762456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:40.762456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:40.762456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:40.762456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:40.762456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:40.762456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:40.762456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:40.763623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:40.763623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:40.763623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:40.763623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:40.763623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:40.763623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:40.763623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:40.763623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:40.763623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:40.763623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:40.763623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:40.763623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:40.763623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:40.763623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:40.763623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:40.763623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:40.763623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:40.764573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:40.764573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:40.764573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:40.764754 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 340/680] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:40.764754 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020942a3-c5bf-463b-9102-7273c1ff4f0d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:40.766118 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 341/681] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:40.767049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfbb08-c4d7-4d84-9322-d53dd89fd678 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:40.842265 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:40.889070 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfbb08-c4d7-4d84-9322-d53dd89fd678 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:40.901940 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020942a3-c5bf-463b-9102-7273c1ff4f0d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:40.932125 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39dd402-7c16-4e7a-80cd-210658ebe290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:40.933047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dd402-7c16-4e7a-80cd-210658ebe290 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:40.933360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dd402-7c16-4e7a-80cd-210658ebe290 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:40.933628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dd402-7c16-4e7a-80cd-210658ebe290 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:40.944362 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014d6e5-9f6b-4b1a-947e-5aa62c2253f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:40.944813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39dd402-7c16-4e7a-80cd-210658ebe290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:40.945259 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014d6e5-9f6b-4b1a-947e-5aa62c2253f0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:40.945662 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014d6e5-9f6b-4b1a-947e-5aa62c2253f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:40.946052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014d6e5-9f6b-4b1a-947e-5aa62c2253f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:40.946898 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014d6e5-9f6b-4b1a-947e-5aa62c2253f0 None admin] RBAC: Adding query filter params (role.id=9aa748c8ea2542e6987e0b93190ea4e6, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=5de1586efc8c4dec898b0fb54b56acf0) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:40.947325 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014d6e5-9f6b-4b1a-947e-5aa62c2253f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:40.949004 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014d6e5-9f6b-4b1a-947e-5aa62c2253f0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:40.949987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-a39dd402-7c16-4e7a-80cd-210658ebe290 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:42:40.949987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:40.949987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:40.949987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:40.949987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:40.949987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:40.949987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:40.949987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:40.949987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:40.949987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:40.949987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:40.949987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:40.949987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:40.949987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:40.949987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:40.949987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:40.951671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:40.951671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:40.951671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:40.951671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:40.951671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:40.951671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:40.951671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:40.951671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:40.951671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:40.951671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:40.951671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:40.951671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:40.951671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:40.951671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:40.951671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:40.951671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:40.953566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:40.953566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:40.953566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:40.953566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:40.953566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:40.953566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:40.953566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:40.953566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:40.953566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:40.953566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:40.953566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:42:40.953566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:40.956132 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 342/682] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:42:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:40.956996 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39dd402-7c16-4e7a-80cd-210658ebe290 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:40.964637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 341/683] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:42:40 2022] GET /identity/v3/role_assignments?user.id=5de1586efc8c4dec898b0fb54b56acf0&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=9aa748c8ea2542e6987e0b93190ea4e6 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:40.967389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014d6e5-9f6b-4b1a-947e-5aa62c2253f0 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:41.007184 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:41.083532 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39dd402-7c16-4e7a-80cd-210658ebe290 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:41.128487 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0940f9-26b2-4df7-9b36-08163b2df30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:41.129017 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0940f9-26b2-4df7-9b36-08163b2df30b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:41.129300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0940f9-26b2-4df7-9b36-08163b2df30b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:41.129572 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0940f9-26b2-4df7-9b36-08163b2df30b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:41.130319 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0940f9-26b2-4df7-9b36-08163b2df30b None admin] RBAC: Adding query filter params (role.id=9aa748c8ea2542e6987e0b93190ea4e6, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=2ee2e93901914b1d82607a3ae0e2acf1) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:41.130685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0940f9-26b2-4df7-9b36-08163b2df30b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:41.132359 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0940f9-26b2-4df7-9b36-08163b2df30b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:41.144131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 343/684] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:42:40 2022] GET /identity/v3/role_assignments?user.id=2ee2e93901914b1d82607a3ae0e2acf1&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=9aa748c8ea2542e6987e0b93190ea4e6 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:41.144131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0940f9-26b2-4df7-9b36-08163b2df30b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:41.146113 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014d6e5-9f6b-4b1a-947e-5aa62c2253f0 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:41.194429 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:41.220419 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2ab45-036c-4baa-8441-39af91bab70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:41.221116 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2ab45-036c-4baa-8441-39af91bab70c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:41.221477 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2ab45-036c-4baa-8441-39af91bab70c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:41.221804 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2ab45-036c-4baa-8441-39af91bab70c None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:41.222613 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f2ab45-036c-4baa-8441-39af91bab70c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:41.222913 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f2ab45-036c-4baa-8441-39af91bab70c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:41.225054 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f2ab45-036c-4baa-8441-39af91bab70c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:41.242037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 342/685] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:41.247911 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2ab45-036c-4baa-8441-39af91bab70c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:41.265232 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0940f9-26b2-4df7-9b36-08163b2df30b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:41.267908 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:41.308133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ea44b-95dc-456a-8cfd-5103a5fd5cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:41.309440 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ea44b-95dc-456a-8cfd-5103a5fd5cdf None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:41.309440 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ea44b-95dc-456a-8cfd-5103a5fd5cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:41.309639 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ea44b-95dc-456a-8cfd-5103a5fd5cdf None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:41.310354 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ea44b-95dc-456a-8cfd-5103a5fd5cdf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:41.310697 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ea44b-95dc-456a-8cfd-5103a5fd5cdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:41.312198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ea44b-95dc-456a-8cfd-5103a5fd5cdf None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:41.324835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 344/686] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:42:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:41.330991 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ea44b-95dc-456a-8cfd-5103a5fd5cdf None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:41.400492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2ab45-036c-4baa-8441-39af91bab70c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:41.476191 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f678c-ebc1-4d84-ac7d-f0f6dfb1ea27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:41.477584 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f678c-ebc1-4d84-ac7d-f0f6dfb1ea27 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:41.478267 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f678c-ebc1-4d84-ac7d-f0f6dfb1ea27 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:41.479700 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f678c-ebc1-4d84-ac7d-f0f6dfb1ea27 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:41.495119 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ea44b-95dc-456a-8cfd-5103a5fd5cdf None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:41.499170 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f678c-ebc1-4d84-ac7d-f0f6dfb1ea27 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:41.502170 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f678c-ebc1-4d84-ac7d-f0f6dfb1ea27 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:41.519329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-b15f678c-ebc1-4d84-ac7d-f0f6dfb1ea27 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:42:41.519329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:41.519329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:41.519329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:41.519329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:41.519329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:41.519329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:41.519329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:41.519329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:41.519329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:41.519329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:41.519329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:41.519329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:41.519329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:41.519329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:41.519329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:41.521359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:41.521359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:41.521359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:41.521359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:41.521359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:41.521359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:41.521359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:41.521359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:41.521359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:41.521359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:41.521359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:41.521359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:41.521359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:41.521359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:41.521359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:41.521359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:41.521359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:41.521359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:41.523837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:41.523837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:41.523837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:41.523837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:41.523837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:41.523837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:41.523837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:41.523837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:41.523837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:42:41.523837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:41.523837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 343/687] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:42:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:42:41.526935 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f678c-ebc1-4d84-ac7d-f0f6dfb1ea27 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:41.584129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:41.587888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d89262-118f-4202-b434-a4263cab532e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:41.589721 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d89262-118f-4202-b434-a4263cab532e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:41.590586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d89262-118f-4202-b434-a4263cab532e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:41.591833 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d89262-118f-4202-b434-a4263cab532e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:41.618258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d89262-118f-4202-b434-a4263cab532e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:41.623438 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d89262-118f-4202-b434-a4263cab532e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:41.657150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-23d89262-118f-4202-b434-a4263cab532e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:41.657150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:41.657150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:41.657150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:41.657150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:41.657150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:41.657150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:41.657150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:41.657150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:41.657150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:41.657150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:41.657150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:41.657150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:41.657150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:41.657150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:41.657150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:41.659169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:41.659169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:41.659169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:41.659169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:41.659169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:41.659169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:41.659169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:41.659169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:41.659169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:41.659169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:41.659169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:41.659169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:41.659169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:41.659169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:41.659169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:41.659169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:41.659169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:41.661192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:41.661192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:41.661192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:41.664700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 345/688] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:41.670395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d89262-118f-4202-b434-a4263cab532e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:41.707289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:41.729693 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f678c-ebc1-4d84-ac7d-f0f6dfb1ea27 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:41.812819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e190447-d152-42ad-bf68-db0cef0bb2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:41.814195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e190447-d152-42ad-bf68-db0cef0bb2f4 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:41.814866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e190447-d152-42ad-bf68-db0cef0bb2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:41.815778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e190447-d152-42ad-bf68-db0cef0bb2f4 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:41.819509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e190447-d152-42ad-bf68-db0cef0bb2f4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:41.820561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e190447-d152-42ad-bf68-db0cef0bb2f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:41.825874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e190447-d152-42ad-bf68-db0cef0bb2f4 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:41.899928 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 344/689] 10.222.0.67 () {56 vars in 1135 bytes} [Wed Aug 24 08:42:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:41.909848 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e190447-d152-42ad-bf68-db0cef0bb2f4 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:41.956038 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d89262-118f-4202-b434-a4263cab532e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:42.029770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392d1cf-9ab2-43a6-a6ac-eb12d2a60c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:42.030404 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392d1cf-9ab2-43a6-a6ac-eb12d2a60c75 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:42.030683 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392d1cf-9ab2-43a6-a6ac-eb12d2a60c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:42.030955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392d1cf-9ab2-43a6-a6ac-eb12d2a60c75 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:42.031617 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392d1cf-9ab2-43a6-a6ac-eb12d2a60c75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:42.031919 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392d1cf-9ab2-43a6-a6ac-eb12d2a60c75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:42.033356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392d1cf-9ab2-43a6-a6ac-eb12d2a60c75 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:42.063161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 346/690] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:42.068664 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392d1cf-9ab2-43a6-a6ac-eb12d2a60c75 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:42.113405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:42.119919 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e190447-d152-42ad-bf68-db0cef0bb2f4 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:42.197815 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78cf542-3402-4cec-9fad-2c8a4e281cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:42.199639 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78cf542-3402-4cec-9fad-2c8a4e281cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:42.200261 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78cf542-3402-4cec-9fad-2c8a4e281cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:42.201890 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78cf542-3402-4cec-9fad-2c8a4e281cf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:42.217675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:42.261822 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392d1cf-9ab2-43a6-a6ac-eb12d2a60c75 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:42.340455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e72f7-4567-4dc7-a80f-929e2f4179a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:42.342349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e72f7-4567-4dc7-a80f-929e2f4179a3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:42.343040 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e72f7-4567-4dc7-a80f-929e2f4179a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:42.343463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e72f7-4567-4dc7-a80f-929e2f4179a3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:42.347604 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78cf542-3402-4cec-9fad-2c8a4e281cf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:42.349348 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78cf542-3402-4cec-9fad-2c8a4e281cf4 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:42.358904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e72f7-4567-4dc7-a80f-929e2f4179a3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:42.363417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e72f7-4567-4dc7-a80f-929e2f4179a3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:42.391338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-728e72f7-4567-4dc7-a80f-929e2f4179a3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:42:42.391338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:42.391338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:42.391338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:42.391338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:42.391338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:42.391338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:42.391338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:42.391338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:42.391338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:42.391338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:42.391338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:42.391338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:42.391338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:42.391338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:42.391338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:42.392563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:42.392563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:42.392563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:42.392563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:42.392563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:42.392563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:42.392563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:42.392563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:42.392563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:42.392563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:42.392563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:42.392563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:42.392563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:42.392563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:42.392563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:42.392563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:42.392563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:42.392563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:42.393767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:42.393767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:42.393767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:42.393767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:42.393767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:42.393767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:42.393767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:42.393767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:42.393767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:42:42.393767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:42.399309 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 347/691] 10.222.0.67 () {56 vars in 1093 bytes} [Wed Aug 24 08:42:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:42.405597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e72f7-4567-4dc7-a80f-929e2f4179a3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:42.547755 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:42.556514 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 345/692] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:42:42.562431 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78cf542-3402-4cec-9fad-2c8a4e281cf4 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:42.607090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e72f7-4567-4dc7-a80f-929e2f4179a3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:42.651772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec04d99-74ef-4bf9-839a-cef6c810dd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:42.652427 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec04d99-74ef-4bf9-839a-cef6c810dd40 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:42.652704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec04d99-74ef-4bf9-839a-cef6c810dd40 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:42.652969 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec04d99-74ef-4bf9-839a-cef6c810dd40 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:42.653769 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec04d99-74ef-4bf9-839a-cef6c810dd40 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:42.654081 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec04d99-74ef-4bf9-839a-cef6c810dd40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:42.655626 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec04d99-74ef-4bf9-839a-cef6c810dd40 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:42.658730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:42.684166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 348/693] 10.222.0.67 () {56 vars in 1133 bytes} [Wed Aug 24 08:42:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:42.685843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d623d6-8709-41b1-951d-e2ca29c6e899 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:42.686137 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d623d6-8709-41b1-951d-e2ca29c6e899 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:42.686412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d623d6-8709-41b1-951d-e2ca29c6e899 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:42.688401 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 349/694] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:42.698741 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b00d4e-d3e9-4d09-aaae-6047c48958c3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:42.699028 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b00d4e-d3e9-4d09-aaae-6047c48958c3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:42.699301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b00d4e-d3e9-4d09-aaae-6047c48958c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:42.713686 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78cf542-3402-4cec-9fad-2c8a4e281cf4 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:42.719397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e7b00d4e-d3e9-4d09-aaae-6047c48958c3 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:42.720901 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e7b00d4e-d3e9-4d09-aaae-6047c48958c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:42.763316 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794c3ee-0d8d-4ebf-b63e-3639997dd62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:42.764083 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794c3ee-0d8d-4ebf-b63e-3639997dd62c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:42.764387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794c3ee-0d8d-4ebf-b63e-3639997dd62c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:42.764656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794c3ee-0d8d-4ebf-b63e-3639997dd62c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:42.777313 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794c3ee-0d8d-4ebf-b63e-3639997dd62c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:42.778840 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794c3ee-0d8d-4ebf-b63e-3639997dd62c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:42.798405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-4794c3ee-0d8d-4ebf-b63e-3639997dd62c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:42.798405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:42.798405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:42.798405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:42.798405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:42.798405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:42.798405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:42.798405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:42.798405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:42.798405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:42.798405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:42.798405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:42.798405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:42.798405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:42.798405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:42.798405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:42.799409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:42.799409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:42.799409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:42.799409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:42.799409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:42.799409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:42.799409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:42.799409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:42.799409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:42.799409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:42.799409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:42.799409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:42.799409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:42.799409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:42.799409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:42.799409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:42.799409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:42.802103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:42.802103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:42.802103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:42.802103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 346/695] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:42:42.802103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794c3ee-0d8d-4ebf-b63e-3639997dd62c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:42.819879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 350/696] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:42.821531 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b00d4e-d3e9-4d09-aaae-6047c48958c3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:42.905754 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:43.011884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b00d4e-d3e9-4d09-aaae-6047c48958c3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:43.013478 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794c3ee-0d8d-4ebf-b63e-3639997dd62c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:43.059083 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:43.076478 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c9c93-a533-4c9f-b80c-9b0ca60124ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:43.077489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c9c93-a533-4c9f-b80c-9b0ca60124ad None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:43.077979 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c9c93-a533-4c9f-b80c-9b0ca60124ad None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:43.078469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c9c93-a533-4c9f-b80c-9b0ca60124ad None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:43.079665 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c9c93-a533-4c9f-b80c-9b0ca60124ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:43.080249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c9c93-a533-4c9f-b80c-9b0ca60124ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:43.083204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c9c93-a533-4c9f-b80c-9b0ca60124ad None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:43.094349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991847b-c548-4c7e-95ae-598a3ed705cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:43.094984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991847b-c548-4c7e-95ae-598a3ed705cf None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:43.095266 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991847b-c548-4c7e-95ae-598a3ed705cf None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:43.095538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991847b-c548-4c7e-95ae-598a3ed705cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:43.099023 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 351/697] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:43.105689 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c9c93-a533-4c9f-b80c-9b0ca60124ad None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:43.240456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c9c93-a533-4c9f-b80c-9b0ca60124ad None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:43.279185 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991847b-c548-4c7e-95ae-598a3ed705cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:43.284502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991847b-c548-4c7e-95ae-598a3ed705cf None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:43.316815 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ae971-49a0-4eac-80f7-6bcd64582566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:43.317877 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ae971-49a0-4eac-80f7-6bcd64582566 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:43.318380 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ae971-49a0-4eac-80f7-6bcd64582566 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:43.318878 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ae971-49a0-4eac-80f7-6bcd64582566 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:43.328427 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ae971-49a0-4eac-80f7-6bcd64582566 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:43.330414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ae971-49a0-4eac-80f7-6bcd64582566 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:43.344342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-322ae971-49a0-4eac-80f7-6bcd64582566 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:43.344342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:43.344342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:43.344342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:43.344342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:43.344342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:43.344342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:43.344342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:43.344342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:43.344342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:43.344342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:43.344342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:43.344342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:43.344342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:43.344342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:43.345405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:43.345405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:43.345405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:43.345405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:43.345405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:43.345405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:43.345405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:43.345405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:43.345405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:43.345405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:43.345405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:43.345405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:43.345405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:43.345405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:43.345405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:43.345405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:43.345405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:43.346484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:43.346484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:43.346484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:43.346484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:43.346484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:43.346484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:43.346484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:43.346484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:43.349461 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 352/698] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:43.351219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ae971-49a0-4eac-80f7-6bcd64582566 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:43.374962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:43.424419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:43.449192 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 347/699] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:42:43.450089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991847b-c548-4c7e-95ae-598a3ed705cf None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:43.510107 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ae971-49a0-4eac-80f7-6bcd64582566 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:43.566139 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ce2c2-5a31-4847-9299-a5151058f6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:43.566892 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ce2c2-5a31-4847-9299-a5151058f6df None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:43.567114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ce2c2-5a31-4847-9299-a5151058f6df None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:43.567549 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ce2c2-5a31-4847-9299-a5151058f6df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:43.587592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ce2c2-5a31-4847-9299-a5151058f6df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:43.590815 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ce2c2-5a31-4847-9299-a5151058f6df None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:43.593080 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991847b-c548-4c7e-95ae-598a3ed705cf None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:43.619274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-c85ce2c2-5a31-4847-9299-a5151058f6df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:43.619274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:43.619274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:43.619274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:43.619274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:43.619274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:43.619274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:43.619274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:43.619274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:43.619274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:43.619274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:43.619274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:43.619274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:43.619274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:43.619274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:43.619274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:43.620356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:43.620356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:43.620356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:43.620356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:43.620356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:43.620356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:43.620356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:43.620356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:43.620356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:43.620356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:43.620356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:43.620356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:43.620356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:43.620356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:43.620356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:43.620356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:43.620356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:43.621244 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:43.621244 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:43.621244 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:43.626577 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 353/700] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:43.627802 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ce2c2-5a31-4847-9299-a5151058f6df None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:43.643036 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272dcdb-7cae-49d1-9be1-1626681fb303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:43.644463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272dcdb-7cae-49d1-9be1-1626681fb303 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:43.645268 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272dcdb-7cae-49d1-9be1-1626681fb303 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:43.645791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272dcdb-7cae-49d1-9be1-1626681fb303 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:43.646745 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272dcdb-7cae-49d1-9be1-1626681fb303 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:43.648066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272dcdb-7cae-49d1-9be1-1626681fb303 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:43.650705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272dcdb-7cae-49d1-9be1-1626681fb303 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:43.669004 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 348/701] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:43.670924 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272dcdb-7cae-49d1-9be1-1626681fb303 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:43.689528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:43.766557 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:43.775147 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ce2c2-5a31-4847-9299-a5151058f6df None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:43.830495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67426d3-2fde-4584-8d65-1604a9a08232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:43.831109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67426d3-2fde-4584-8d65-1604a9a08232 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:43.831284 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272dcdb-7cae-49d1-9be1-1626681fb303 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:43.831414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67426d3-2fde-4584-8d65-1604a9a08232 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:43.831687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67426d3-2fde-4584-8d65-1604a9a08232 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:43.845140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67426d3-2fde-4584-8d65-1604a9a08232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:43.846627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67426d3-2fde-4584-8d65-1604a9a08232 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:43.874698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f67426d3-2fde-4584-8d65-1604a9a08232 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:43.874698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:43.874698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:43.874698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:43.874698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:43.874698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:43.874698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:43.874698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:43.874698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:43.874698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:43.874698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:43.874698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:43.874698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:43.874698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:43.874698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:43.874698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:43.875773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:43.875773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:43.875773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:43.875773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:43.875773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:43.875773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:43.875773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:43.875773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:43.875773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:43.875773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:43.875773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:43.875773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:43.875773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:43.875773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:43.875773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:43.875773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:43.875773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:43.876772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:43.876772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:43.876772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:43.883664 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 354/702] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:43.884950 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67426d3-2fde-4584-8d65-1604a9a08232 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:43.909930 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36bea5-7b1f-44bd-a69c-a50fbbd34ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:43.910701 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36bea5-7b1f-44bd-a69c-a50fbbd34ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:43.911071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36bea5-7b1f-44bd-a69c-a50fbbd34ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:43.911512 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36bea5-7b1f-44bd-a69c-a50fbbd34ef8 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:43.912535 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36bea5-7b1f-44bd-a69c-a50fbbd34ef8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:43.913555 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36bea5-7b1f-44bd-a69c-a50fbbd34ef8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:43.915603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36bea5-7b1f-44bd-a69c-a50fbbd34ef8 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:43.937613 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 349/703] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:43.937613 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7789c-4472-41d1-a203-fd34f3e01dc0 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:43.937613 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7789c-4472-41d1-a203-fd34f3e01dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:43.937613 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7789c-4472-41d1-a203-fd34f3e01dc0 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:43.937613 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 350/704] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:43.939351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7789c-4472-41d1-a203-fd34f3e01dc0 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:44.020256 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67426d3-2fde-4584-8d65-1604a9a08232 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:44.025052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:44.057786 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f4e13-7687-4f89-a86c-5be3650b5167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:44.058235 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7789c-4472-41d1-a203-fd34f3e01dc0 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:44.058383 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f4e13-7687-4f89-a86c-5be3650b5167 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:44.058663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f4e13-7687-4f89-a86c-5be3650b5167 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:44.058930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f4e13-7687-4f89-a86c-5be3650b5167 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:44.059658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f4e13-7687-4f89-a86c-5be3650b5167 None admin] RBAC: Adding query filter params (role.id=9aa748c8ea2542e6987e0b93190ea4e6, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=46b1417d4ac24c8982b2052f4e99042a) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:44.059963 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f4e13-7687-4f89-a86c-5be3650b5167 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:44.061446 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f4e13-7687-4f89-a86c-5be3650b5167 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:44.067809 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 355/705] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:42:43 2022] GET /identity/v3/role_assignments?user.id=46b1417d4ac24c8982b2052f4e99042a&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=9aa748c8ea2542e6987e0b93190ea4e6 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:44.069660 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99687e5-186e-4785-a316-d0b1a484e5b9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:44.069992 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99687e5-186e-4785-a316-d0b1a484e5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:44.070215 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99687e5-186e-4785-a316-d0b1a484e5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:44.091482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e99687e5-186e-4785-a316-d0b1a484e5b9 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:44.092731 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e99687e5-186e-4785-a316-d0b1a484e5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:44.102782 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:44.103768 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473d95b-a7ce-44d3-a81b-d6431ef490d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:44.104466 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473d95b-a7ce-44d3-a81b-d6431ef490d2 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:44.104746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473d95b-a7ce-44d3-a81b-d6431ef490d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:44.105023 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473d95b-a7ce-44d3-a81b-d6431ef490d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:44.105683 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473d95b-a7ce-44d3-a81b-d6431ef490d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:44.105989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473d95b-a7ce-44d3-a81b-d6431ef490d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:44.107685 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473d95b-a7ce-44d3-a81b-d6431ef490d2 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:44.128562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 351/706] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:44.128562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473d95b-a7ce-44d3-a81b-d6431ef490d2 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:44.201847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 356/707] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:44.201847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2595a-023f-4d47-91cb-03eef356601c None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:44.201847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2595a-023f-4d47-91cb-03eef356601c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:44.201847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2595a-023f-4d47-91cb-03eef356601c None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:44.201847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 357/708] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:44.201847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2595a-023f-4d47-91cb-03eef356601c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:44.282293 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473d95b-a7ce-44d3-a81b-d6431ef490d2 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:44.341892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:44.350357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91caf1-552f-4d60-a385-262bd832ae39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:44.351926 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91caf1-552f-4d60-a385-262bd832ae39 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:44.352599 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91caf1-552f-4d60-a385-262bd832ae39 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:44.353110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91caf1-552f-4d60-a385-262bd832ae39 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:44.354065 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91caf1-552f-4d60-a385-262bd832ae39 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:44.355764 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91caf1-552f-4d60-a385-262bd832ae39 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:44.357774 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2595a-023f-4d47-91cb-03eef356601c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:44.382549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 352/709] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 24 08:42:44 2022] POST /identity/v3/users => generated 254 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:42:44.383735 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ee9da-1150-4f28-ac74-cd146879c8a6 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:44.384016 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ee9da-1150-4f28-ac74-cd146879c8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:44.384323 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ee9da-1150-4f28-ac74-cd146879c8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:44.412032 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e68ee9da-1150-4f28-ac74-cd146879c8a6 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:44.413930 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e68ee9da-1150-4f28-ac74-cd146879c8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:44.422459 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc73f6e-e8d5-4320-b5ff-b56563508179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:44.423115 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc73f6e-e8d5-4320-b5ff-b56563508179 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:44.423387 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc73f6e-e8d5-4320-b5ff-b56563508179 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:44.423671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc73f6e-e8d5-4320-b5ff-b56563508179 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:44.443637 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc73f6e-e8d5-4320-b5ff-b56563508179 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:44.445734 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc73f6e-e8d5-4320-b5ff-b56563508179 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:44.458174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-0cc73f6e-e8d5-4320-b5ff-b56563508179 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:44.458174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:44.458174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:44.458174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:44.458174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:44.458174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:44.458174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:44.458174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:44.458174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:44.458174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:44.458174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:44.458174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:44.458174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:44.458174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:44.458174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:44.459201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:44.459201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:44.459201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:44.459201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:44.459201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:44.459201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:44.459201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:44.459201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:44.459201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:44.459201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:44.459201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:44.459201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:44.459201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:44.459201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:44.459201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:44.459201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:44.459201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:44.461382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:44.461382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:44.461382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:44.461382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:44.461382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:44.461382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:44.461382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:44.461382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:44.461382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 358/710] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:44.466432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc73f6e-e8d5-4320-b5ff-b56563508179 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:44.478695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:44.536734 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 353/711] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:44.601831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ee9da-1150-4f28-ac74-cd146879c8a6 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:44.612548 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc73f6e-e8d5-4320-b5ff-b56563508179 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:44.658833 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19122f06-7e1f-441d-be2a-81674c880c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:44.659454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19122f06-7e1f-441d-be2a-81674c880c2b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:44.659865 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19122f06-7e1f-441d-be2a-81674c880c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:44.660183 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19122f06-7e1f-441d-be2a-81674c880c2b None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:44.660913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19122f06-7e1f-441d-be2a-81674c880c2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:44.661215 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19122f06-7e1f-441d-be2a-81674c880c2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:44.663468 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19122f06-7e1f-441d-be2a-81674c880c2b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:44.677710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 359/712] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:44.685285 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19122f06-7e1f-441d-be2a-81674c880c2b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:44.738283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ee9da-1150-4f28-ac74-cd146879c8a6 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:44.762279 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:44.762811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:44.790301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc3d7a-2c29-4c67-8d21-7fa5155db970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:44.791490 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc3d7a-2c29-4c67-8d21-7fa5155db970 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:44.792004 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc3d7a-2c29-4c67-8d21-7fa5155db970 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:44.792752 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc3d7a-2c29-4c67-8d21-7fa5155db970 None admin] PATH_INFO: `/v3/services/network` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:44.793950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc3d7a-2c29-4c67-8d21-7fa5155db970 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:44.796837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc3d7a-2c29-4c67-8d21-7fa5155db970 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:44.816323 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19122f06-7e1f-441d-be2a-81674c880c2b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:44.822014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-21dc3d7a-2c29-4c67-8d21-7fa5155db970 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:42:44.822014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:44.822014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:42:44.822014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:42:44.822014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:42:44.822014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:42:44.822014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:42:44.822014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:42:44.822014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:42:44.822014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:44.822014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:42:44.822014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:44.822014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:44.822014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:44.822014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:44.822014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:44.822014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:44.823076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:44.823076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:44.823076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:44.823076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:44.823076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:44.823076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:44.823076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:42:44.823076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:42:44.823076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:42:44.823076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:42:44.823076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:44.823076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:44.823076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:44.823076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:44.823076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:44.823076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:44.823914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:44.823914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:44.823914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:44.823914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:44.823914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:44.823914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:44.823914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:44.823914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:44.823914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:44.823914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:44.823914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:42:44.823914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:42:44.823914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:42:44.823914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:44.825537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 354/713] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:44 2022] GET /identity/v3/services/network => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:44.831099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc3d7a-2c29-4c67-8d21-7fa5155db970 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:44.867786 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3adccdd-19a5-43ee-ba4c-7ffb0a04d1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:44.868492 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adccdd-19a5-43ee-ba4c-7ffb0a04d1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:44.868774 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adccdd-19a5-43ee-ba4c-7ffb0a04d1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:44.869304 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adccdd-19a5-43ee-ba4c-7ffb0a04d1c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:44.870123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3adccdd-19a5-43ee-ba4c-7ffb0a04d1c0 None admin] RBAC: Adding query filter params (role.id=b13ffa72344c4856901b1ba003e273fd, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=ca56dd7216fb4946946b2801e144fec2) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:44.870431 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3adccdd-19a5-43ee-ba4c-7ffb0a04d1c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:44.872838 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3adccdd-19a5-43ee-ba4c-7ffb0a04d1c0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:44.880525 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 360/714] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:42:44 2022] GET /identity/v3/role_assignments?user.id=ca56dd7216fb4946946b2801e144fec2&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=b13ffa72344c4856901b1ba003e273fd => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:44.910835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3adccdd-19a5-43ee-ba4c-7ffb0a04d1c0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:44.995760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc3d7a-2c29-4c67-8d21-7fa5155db970 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:45.063305 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3adccdd-19a5-43ee-ba4c-7ffb0a04d1c0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:45.071784 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361eb76-110b-46a1-9399-8e4e14017b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:45.072440 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361eb76-110b-46a1-9399-8e4e14017b61 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:45.072719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361eb76-110b-46a1-9399-8e4e14017b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:45.072980 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361eb76-110b-46a1-9399-8e4e14017b61 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:45.073619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361eb76-110b-46a1-9399-8e4e14017b61 None admin] RBAC: Adding query filter params (name=network) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:45.073916 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361eb76-110b-46a1-9399-8e4e14017b61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:45.075308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361eb76-110b-46a1-9399-8e4e14017b61 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:45.080561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 355/715] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 24 08:42:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:45.086588 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361eb76-110b-46a1-9399-8e4e14017b61 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:45.118134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:45.146208 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:45.156847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccca93a-72d8-4998-942e-1fccd18536dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:45.157503 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccca93a-72d8-4998-942e-1fccd18536dc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:45.157779 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccca93a-72d8-4998-942e-1fccd18536dc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:45.158046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccca93a-72d8-4998-942e-1fccd18536dc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:45.158636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccca93a-72d8-4998-942e-1fccd18536dc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:45.160003 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccca93a-72d8-4998-942e-1fccd18536dc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:45.190592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-cccca93a-72d8-4998-942e-1fccd18536dc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:42:45.190592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:45.190592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:42:45.190592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:42:45.190592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:42:45.190592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:42:45.190592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:42:45.190592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:42:45.190592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:42:45.190592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:45.190592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:42:45.190592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:45.190592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:45.190592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:45.190592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:45.190592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:45.190592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:45.191648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:45.191648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:45.191648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:45.191648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:45.191648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:45.191648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:45.191648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:42:45.191648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:42:45.191648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:42:45.191648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:42:45.191648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:45.191648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:45.191648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:45.191648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:45.191648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:45.191648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:45.192527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:45.192527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:45.192527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:45.192527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:45.192527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:45.192527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:45.192527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:45.192527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:45.192527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:45.192527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:45.192527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:42:45.192527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:42:45.192527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:42:45.192527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:45.192527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 361/716] 10.222.0.67 () {56 vars in 1129 bytes} [Wed Aug 24 08:42:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:42:45.198195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361eb76-110b-46a1-9399-8e4e14017b61 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:45.206181 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccca93a-72d8-4998-942e-1fccd18536dc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:45.237142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e11c8-437e-4fe6-b58a-b59829847876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:45.237763 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e11c8-437e-4fe6-b58a-b59829847876 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:45.238052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e11c8-437e-4fe6-b58a-b59829847876 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:45.238321 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e11c8-437e-4fe6-b58a-b59829847876 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:45.238976 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e11c8-437e-4fe6-b58a-b59829847876 None admin] RBAC: Adding query filter params (type=network) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:45.239438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e11c8-437e-4fe6-b58a-b59829847876 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:45.240973 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e11c8-437e-4fe6-b58a-b59829847876 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:45.247835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 356/717] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 24 08:42:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:45.399096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccca93a-72d8-4998-942e-1fccd18536dc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:45.449641 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58079d94-b737-4d57-a8b7-7473a41c0612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:45.450623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58079d94-b737-4d57-a8b7-7473a41c0612 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:45.451141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58079d94-b737-4d57-a8b7-7473a41c0612 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:45.451613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58079d94-b737-4d57-a8b7-7473a41c0612 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:45.452481 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58079d94-b737-4d57-a8b7-7473a41c0612 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:45.452996 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58079d94-b737-4d57-a8b7-7473a41c0612 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:45.454583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58079d94-b737-4d57-a8b7-7473a41c0612 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:45.462442 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 362/718] 10.222.0.67 () {56 vars in 1124 bytes} [Wed Aug 24 08:42:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:45.468451 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e11c8-437e-4fe6-b58a-b59829847876 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:45.588197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e11c8-437e-4fe6-b58a-b59829847876 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:45.598541 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:45.625850 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9cf6d-2ad9-43d0-8763-5f90a79ccf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:45.626743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9cf6d-2ad9-43d0-8763-5f90a79ccf68 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:45.627022 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9cf6d-2ad9-43d0-8763-5f90a79ccf68 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:45.627291 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9cf6d-2ad9-43d0-8763-5f90a79ccf68 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:45.627926 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9cf6d-2ad9-43d0-8763-5f90a79ccf68 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:45.628253 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9cf6d-2ad9-43d0-8763-5f90a79ccf68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:45.630199 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9cf6d-2ad9-43d0-8763-5f90a79ccf68 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:45.639745 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 357/719] 10.222.0.67 () {56 vars in 1124 bytes} [Wed Aug 24 08:42:45 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:46.484499 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571d6db-6971-47c2-af8d-4c7aa8d1f396 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:46.485035 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571d6db-6971-47c2-af8d-4c7aa8d1f396 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:46.485035 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571d6db-6971-47c2-af8d-4c7aa8d1f396 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:46.486592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 363/720] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:46.493591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b49b11-cc6c-4f07-82ac-e0192f6af348 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:46.494138 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b49b11-cc6c-4f07-82ac-e0192f6af348 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:46.494551 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b49b11-cc6c-4f07-82ac-e0192f6af348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:46.541013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f1b49b11-cc6c-4f07-82ac-e0192f6af348 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:46.542983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f1b49b11-cc6c-4f07-82ac-e0192f6af348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:46.670299 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 358/721] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 08:42:47.051319 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571d6db-6971-47c2-af8d-4c7aa8d1f396 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:47.123129 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:47.185866 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571d6db-6971-47c2-af8d-4c7aa8d1f396 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:47.241606 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d376d48-d709-44af-8175-a2b24cebbbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:47.242308 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d376d48-d709-44af-8175-a2b24cebbbb4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:47.242603 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d376d48-d709-44af-8175-a2b24cebbbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:47.242986 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d376d48-d709-44af-8175-a2b24cebbbb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:47.259672 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d376d48-d709-44af-8175-a2b24cebbbb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:47.264469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-0d376d48-d709-44af-8175-a2b24cebbbb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:42:47.264469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:47.264469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:47.264469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:47.264469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:47.264469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:47.264469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:47.264469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:47.264469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:47.264469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:47.264469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:47.264469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:47.264469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:47.264469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:47.264469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:47.264469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:47.265622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:47.265622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:47.265622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:47.265622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:47.265622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:47.265622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:47.265622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:47.265622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:47.265622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:47.265622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:47.265622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:47.265622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:47.265622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:47.265622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:47.265622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:47.265622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:47.267154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:47.267154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:47.267154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:47.267154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:47.267154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:47.267154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:47.267154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:47.267154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:47.267154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:47.267154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:47.267154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:42:47.267154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:47.267154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 364/722] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:42:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:47.271694 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b49b11-cc6c-4f07-82ac-e0192f6af348 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:47.364062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf954d-d22c-4013-ab6a-2f49004f9c75 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:47.364355 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf954d-d22c-4013-ab6a-2f49004f9c75 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:47.364684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf954d-d22c-4013-ab6a-2f49004f9c75 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:47.366843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 365/723] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:42:47.372181 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb7028-bf31-410b-a057-1dcc1b3eb448 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:47.372555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb7028-bf31-410b-a057-1dcc1b3eb448 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:47.372760 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb7028-bf31-410b-a057-1dcc1b3eb448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:47.399982 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b49b11-cc6c-4f07-82ac-e0192f6af348 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:47.401820 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:47.406237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0fcb7028-bf31-410b-a057-1dcc1b3eb448 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:47.407056 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0fcb7028-bf31-410b-a057-1dcc1b3eb448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:47.452320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df063ed-1658-4b07-9255-d1aba4ad9c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:47.453341 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df063ed-1658-4b07-9255-d1aba4ad9c97 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:47.453848 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df063ed-1658-4b07-9255-d1aba4ad9c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:47.455006 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df063ed-1658-4b07-9255-d1aba4ad9c97 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:47.455901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df063ed-1658-4b07-9255-d1aba4ad9c97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:47.456435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df063ed-1658-4b07-9255-d1aba4ad9c97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:47.458901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df063ed-1658-4b07-9255-d1aba4ad9c97 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:47.472322 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 359/724] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:42:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:47.474780 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa13b7-c2ce-4520-8787-e0760f226246 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:47.476105 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa13b7-c2ce-4520-8787-e0760f226246 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:47.476630 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa13b7-c2ce-4520-8787-e0760f226246 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:47.479393 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 360/725] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:47.481001 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa13b7-c2ce-4520-8787-e0760f226246 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:47.485289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 366/726] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:47.487024 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686aa02e-34e0-4083-8c9c-a467c40482cc None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:47.487024 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686aa02e-34e0-4083-8c9c-a467c40482cc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:47.487232 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686aa02e-34e0-4083-8c9c-a467c40482cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:47.505931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-686aa02e-34e0-4083-8c9c-a467c40482cc None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:47.507655 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-686aa02e-34e0-4083-8c9c-a467c40482cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:47.557369 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:47.646399 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 367/727] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:47.690681 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa13b7-c2ce-4520-8787-e0760f226246 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:47.763145 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2b9d9-919a-42f8-b8fa-cc06dc33be16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:47.764306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2b9d9-919a-42f8-b8fa-cc06dc33be16 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:47.764779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2b9d9-919a-42f8-b8fa-cc06dc33be16 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:47.765240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2b9d9-919a-42f8-b8fa-cc06dc33be16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:47.786153 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d2b9d9-919a-42f8-b8fa-cc06dc33be16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:47.789966 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d2b9d9-919a-42f8-b8fa-cc06dc33be16 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:47.818538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-01d2b9d9-919a-42f8-b8fa-cc06dc33be16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:47.818538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:47.818538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:47.818538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:47.818538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:47.818538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:47.818538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:47.818538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:47.818538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:47.818538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:47.818538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:47.818538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:47.818538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:47.818538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:47.818538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:47.818538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:47.820238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:47.820238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:47.820238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:47.820238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:47.820238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:47.820238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:47.820238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:47.820238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:47.820238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:47.820238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:47.820238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:47.820238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:47.820238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:47.820238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:47.820238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:47.820238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:47.820238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:47.821138 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:47.821138 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:47.821138 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:47.821860 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 361/728] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:47.830226 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686aa02e-34e0-4083-8c9c-a467c40482cc None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:48.051722 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686aa02e-34e0-4083-8c9c-a467c40482cc None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:48.115423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715b775-2e7b-418e-aa00-b374dd213f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:48.116865 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715b775-2e7b-418e-aa00-b374dd213f49 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:48.117317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715b775-2e7b-418e-aa00-b374dd213f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:48.117796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715b775-2e7b-418e-aa00-b374dd213f49 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:48.118915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715b775-2e7b-418e-aa00-b374dd213f49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:48.119380 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715b775-2e7b-418e-aa00-b374dd213f49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:48.121749 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715b775-2e7b-418e-aa00-b374dd213f49 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:48.151562 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 368/729] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:48.159546 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2b9d9-919a-42f8-b8fa-cc06dc33be16 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:48.177854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf642c0-baab-47d5-b14b-b6b4c101afe1 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:48.178326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf642c0-baab-47d5-b14b-b6b4c101afe1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:48.179328 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf642c0-baab-47d5-b14b-b6b4c101afe1 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:48.181928 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 369/730] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:48.188455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cc2de-c391-4074-8b27-0d1d31b930ca None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:48.188872 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cc2de-c391-4074-8b27-0d1d31b930ca None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:48.189329 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cc2de-c391-4074-8b27-0d1d31b930ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:48.219537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:48.233887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ab5cc2de-c391-4074-8b27-0d1d31b930ca None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:48.236493 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ab5cc2de-c391-4074-8b27-0d1d31b930ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:48.274217 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:48.332336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 370/731] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:48.332336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5cc2de-c391-4074-8b27-0d1d31b930ca None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:48.348235 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2b9d9-919a-42f8-b8fa-cc06dc33be16 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:48.398511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf18693-f746-4be1-b25a-95d1f385ec45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:48.399733 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf18693-f746-4be1-b25a-95d1f385ec45 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:48.400213 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf18693-f746-4be1-b25a-95d1f385ec45 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:48.400637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf18693-f746-4be1-b25a-95d1f385ec45 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:48.411145 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf18693-f746-4be1-b25a-95d1f385ec45 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:48.413679 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf18693-f746-4be1-b25a-95d1f385ec45 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:48.428622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-3cf18693-f746-4be1-b25a-95d1f385ec45 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:42:48.428622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:48.428622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:48.428622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:48.428622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:48.428622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:48.428622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:48.428622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:48.428622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:48.428622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:48.428622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:48.428622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:48.428622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:48.428622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:48.428622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:48.428622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:48.429722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:48.429722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:48.429722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:48.429722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:48.429722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:48.429722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:48.429722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:48.429722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:48.429722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:48.429722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:48.429722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:48.429722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:48.429722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:48.429722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:48.429722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:48.429722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:48.429722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:48.429722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:48.430901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:48.430901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:48.430901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:48.430901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:48.430901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:48.430901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:48.430901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:48.430901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:48.430901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:42:48.430901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:48.430901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 362/732] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:42:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:48.431688 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf18693-f746-4be1-b25a-95d1f385ec45 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:48.463663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5cc2de-c391-4074-8b27-0d1d31b930ca None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:48.501678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f8cb4-a370-43cc-8751-2ff61b8d51ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:48.503081 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f8cb4-a370-43cc-8751-2ff61b8d51ce None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:48.503574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f8cb4-a370-43cc-8751-2ff61b8d51ce None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:48.504052 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f8cb4-a370-43cc-8751-2ff61b8d51ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:48.516520 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f8cb4-a370-43cc-8751-2ff61b8d51ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:48.520514 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:48.521955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-305f8cb4-a370-43cc-8751-2ff61b8d51ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:48.521955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:48.521955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:48.521955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:48.521955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:48.521955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:48.521955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:48.521955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:48.521955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:48.521955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:48.521955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:48.521955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:48.521955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:48.521955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:48.521955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:48.521955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:48.524011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:48.524011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:48.524011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:48.524011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:48.524011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:48.524011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:48.524011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:48.524011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:48.524011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:48.524011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:48.524011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:48.524011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:48.524011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:48.524011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:48.524011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:48.524011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:48.524899 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:48.524899 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:48.524899 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:48.524899 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:48.524899 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:48.524899 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:48.524899 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:48.524899 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:48.524899 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:48.524899 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:48.524899 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:48.524899 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:48.527697 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 371/733] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:48.528875 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f8cb4-a370-43cc-8751-2ff61b8d51ce None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:48.548273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf18693-f746-4be1-b25a-95d1f385ec45 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:48.557396 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:48.593628 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67ff4f-7954-4b54-9c3a-6181f0928401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:48.594412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67ff4f-7954-4b54-9c3a-6181f0928401 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:48.594894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67ff4f-7954-4b54-9c3a-6181f0928401 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:48.595446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67ff4f-7954-4b54-9c3a-6181f0928401 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:48.608854 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e67ff4f-7954-4b54-9c3a-6181f0928401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:48.614178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-5e67ff4f-7954-4b54-9c3a-6181f0928401 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:42:48.614178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:48.614178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:48.614178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:48.614178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:48.614178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:48.614178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:48.614178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:48.614178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:48.614178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:48.614178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:48.614178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:48.614178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:48.614178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:48.614178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:48.614178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:48.615350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:48.615350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:48.615350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:48.615350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:48.615350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:48.615350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:48.615350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:48.615350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:48.615350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:48.615350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:48.615350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:48.615350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:48.615350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:48.615350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:48.615350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:48.615350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:48.616278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:48.616278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:48.616278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:48.616278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:48.616278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:48.616278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:48.616278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:48.616278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:48.616278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:48.616278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:48.616278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:42:48.616278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:48.616278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 363/734] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:42:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:48.616278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67ff4f-7954-4b54-9c3a-6181f0928401 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:48.649933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f8cb4-a370-43cc-8751-2ff61b8d51ce None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:48.690721 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb8340-17ee-456e-a41f-433ab0ff843c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:48.691457 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb8340-17ee-456e-a41f-433ab0ff843c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:48.691723 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb8340-17ee-456e-a41f-433ab0ff843c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:48.691997 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb8340-17ee-456e-a41f-433ab0ff843c None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:48.692773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb8340-17ee-456e-a41f-433ab0ff843c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:48.693077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb8340-17ee-456e-a41f-433ab0ff843c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:48.697240 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb8340-17ee-456e-a41f-433ab0ff843c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:48.723603 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 372/735] 10.222.0.67 () {56 vars in 1135 bytes} [Wed Aug 24 08:42:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:48.725364 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f82e4e-e969-4cf1-b4b3-14f130b6f797 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:48.725656 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f82e4e-e969-4cf1-b4b3-14f130b6f797 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:48.725930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f82e4e-e969-4cf1-b4b3-14f130b6f797 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:48.727623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 373/736] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:48.728317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f82e4e-e969-4cf1-b4b3-14f130b6f797 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:48.747846 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67ff4f-7954-4b54-9c3a-6181f0928401 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:48.786370 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c31ead-3230-4030-96db-98ec39f2b6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:48.786999 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c31ead-3230-4030-96db-98ec39f2b6ff None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:48.787301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c31ead-3230-4030-96db-98ec39f2b6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:48.787608 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c31ead-3230-4030-96db-98ec39f2b6ff None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:48.788426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c31ead-3230-4030-96db-98ec39f2b6ff None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:48.788603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c31ead-3230-4030-96db-98ec39f2b6ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:48.789952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c31ead-3230-4030-96db-98ec39f2b6ff None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:48.792616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:48.802353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 364/737] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:42:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:48.804061 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c31ead-3230-4030-96db-98ec39f2b6ff None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:48.839592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:48.862440 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f82e4e-e969-4cf1-b4b3-14f130b6f797 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:48.902613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940c8c4-f0a4-4c55-8e8e-94ffefacc521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:48.903338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940c8c4-f0a4-4c55-8e8e-94ffefacc521 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:48.903617 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940c8c4-f0a4-4c55-8e8e-94ffefacc521 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:48.903886 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940c8c4-f0a4-4c55-8e8e-94ffefacc521 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:48.904596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940c8c4-f0a4-4c55-8e8e-94ffefacc521 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:48.904896 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940c8c4-f0a4-4c55-8e8e-94ffefacc521 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:48.906266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940c8c4-f0a4-4c55-8e8e-94ffefacc521 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:48.915124 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 374/738] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:42:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:48.916413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd9f6a-4a12-45e1-8c44-729834683321 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:48.916702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd9f6a-4a12-45e1-8c44-729834683321 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:48.916980 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd9f6a-4a12-45e1-8c44-729834683321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:48.937036 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-20fd9f6a-4a12-45e1-8c44-729834683321 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:48.938245 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-20fd9f6a-4a12-45e1-8c44-729834683321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:48.940020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c31ead-3230-4030-96db-98ec39f2b6ff None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:48.977090 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714fd6b-f730-4644-b77c-2e0af181fff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:48.977727 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714fd6b-f730-4644-b77c-2e0af181fff6 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:48.977998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714fd6b-f730-4644-b77c-2e0af181fff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:48.978270 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714fd6b-f730-4644-b77c-2e0af181fff6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:49.017826 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 375/739] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 08:42:49.018678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd9f6a-4a12-45e1-8c44-729834683321 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:49.071458 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:49.078408 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714fd6b-f730-4644-b77c-2e0af181fff6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:49.079887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714fd6b-f730-4644-b77c-2e0af181fff6 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:49.116009 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:49.153806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd9f6a-4a12-45e1-8c44-729834683321 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:49.214018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2f95c-fe6f-4736-96f4-1c0b76402e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:49.215473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2f95c-fe6f-4736-96f4-1c0b76402e16 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:49.216209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2f95c-fe6f-4736-96f4-1c0b76402e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:49.216970 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2f95c-fe6f-4736-96f4-1c0b76402e16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:49.234518 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd2f95c-fe6f-4736-96f4-1c0b76402e16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:49.237354 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd2f95c-fe6f-4736-96f4-1c0b76402e16 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:49.250370 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 365/740] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 08:42:49.251832 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714fd6b-f730-4644-b77c-2e0af181fff6 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:49.265590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-7bd2f95c-fe6f-4736-96f4-1c0b76402e16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:49.265590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:49.265590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:49.265590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:49.265590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:49.265590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:49.265590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:49.265590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:49.265590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:49.265590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:49.265590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:49.265590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:49.265590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:49.265590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:49.265590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:49.265590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:49.266616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:49.266616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:49.266616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:49.266616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:49.266616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:49.266616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:49.266616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:49.266616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:49.266616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:49.266616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:49.266616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:49.266616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:49.266616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:49.266616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:49.266616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:49.266616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:49.266616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:49.267544 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:49.267544 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:49.267544 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:49.269308 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 376/741] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:49.270340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2f95c-fe6f-4736-96f4-1c0b76402e16 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:49.381755 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:49.385832 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714fd6b-f730-4644-b77c-2e0af181fff6 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:49.401623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2f95c-fe6f-4736-96f4-1c0b76402e16 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:49.424283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff2a3f-d809-4192-a023-df1412aaf920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:49.424894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff2a3f-d809-4192-a023-df1412aaf920 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:49.425172 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff2a3f-d809-4192-a023-df1412aaf920 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:49.425441 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff2a3f-d809-4192-a023-df1412aaf920 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:49.426346 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff2a3f-d809-4192-a023-df1412aaf920 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:49.427705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff2a3f-d809-4192-a023-df1412aaf920 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:49.429333 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:49.440435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca70a0-a1a3-43aa-9aa3-accd25ad0c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:49.441069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca70a0-a1a3-43aa-9aa3-accd25ad0c4c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:49.441341 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca70a0-a1a3-43aa-9aa3-accd25ad0c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:49.441606 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca70a0-a1a3-43aa-9aa3-accd25ad0c4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:49.443623 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 366/742] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 24 08:42:49 2022] POST /identity/v3/services => generated 240 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:42:49.444521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff2a3f-d809-4192-a023-df1412aaf920 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:49.458189 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ca70a0-a1a3-43aa-9aa3-accd25ad0c4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:49.460658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ca70a0-a1a3-43aa-9aa3-accd25ad0c4c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:49.483778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f4ca70a0-a1a3-43aa-9aa3-accd25ad0c4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:49.483778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:49.483778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:49.483778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:49.483778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:49.483778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:49.483778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:49.483778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:49.483778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:49.483778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:49.483778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:49.483778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:49.483778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:49.483778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:49.483778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:49.483778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:49.488018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:49.488018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:49.488018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:49.488018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:49.488018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:49.488018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:49.488018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:49.488018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:49.488018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:49.488018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:49.488018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:49.488018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:49.488018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:49.488018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:49.488018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:49.488018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:49.488018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:49.490204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:49.490204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:49.490204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:49.490204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 377/743] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:49.490204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca70a0-a1a3-43aa-9aa3-accd25ad0c4c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:49.600047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff2a3f-d809-4192-a023-df1412aaf920 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:49.637736 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca70a0-a1a3-43aa-9aa3-accd25ad0c4c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:49.648661 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b07f5c-555c-401e-867f-be8bbb440703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:49.649325 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b07f5c-555c-401e-867f-be8bbb440703 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:49.649601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b07f5c-555c-401e-867f-be8bbb440703 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:49.649876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b07f5c-555c-401e-867f-be8bbb440703 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:49.664200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b07f5c-555c-401e-867f-be8bbb440703 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:49.666857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b07f5c-555c-401e-867f-be8bbb440703 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:49.680319 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49567b9f-8268-4c71-b19f-5849f4fd9a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:49.680938 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49567b9f-8268-4c71-b19f-5849f4fd9a4c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:49.681213 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49567b9f-8268-4c71-b19f-5849f4fd9a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:49.681489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49567b9f-8268-4c71-b19f-5849f4fd9a4c None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:49.682138 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49567b9f-8268-4c71-b19f-5849f4fd9a4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:49.682438 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49567b9f-8268-4c71-b19f-5849f4fd9a4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:49.683815 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49567b9f-8268-4c71-b19f-5849f4fd9a4c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:49.684071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-68b07f5c-555c-401e-867f-be8bbb440703 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:49.684071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:49.684071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:49.684071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:49.684071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:49.684071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:49.684071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:49.684071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:49.684071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:49.684071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:49.684071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:49.684071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:49.684071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:49.684071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:49.684071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:49.684071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:49.685422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:49.685422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:49.685422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:49.685422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:49.685422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:49.685422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:49.685422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:49.685422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:49.685422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:49.685422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:49.685422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:49.685422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:49.685422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:49.685422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:49.685422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:49.685422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:49.685422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:49.686788 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:49.686788 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:49.686788 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:49.686788 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 367/744] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:49.687220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b07f5c-555c-401e-867f-be8bbb440703 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:49.706485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:49.712221 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 378/745] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:49.712221 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49567b9f-8268-4c71-b19f-5849f4fd9a4c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:49.769362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:49.849295 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b07f5c-555c-401e-867f-be8bbb440703 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:49.880268 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49567b9f-8268-4c71-b19f-5849f4fd9a4c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:49.895580 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62f48e-305d-4d33-901b-a6d229cf0f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:49.896152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62f48e-305d-4d33-901b-a6d229cf0f1d None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:49.896434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62f48e-305d-4d33-901b-a6d229cf0f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:49.896698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62f48e-305d-4d33-901b-a6d229cf0f1d None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:49.897928 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62f48e-305d-4d33-901b-a6d229cf0f1d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:49.899333 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62f48e-305d-4d33-901b-a6d229cf0f1d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:49.920983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 368/746] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 24 08:42:49 2022] POST /identity/v3/services => generated 251 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:42:49.923060 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62f48e-305d-4d33-901b-a6d229cf0f1d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:49.946769 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40733c72-7492-499e-a7e6-23cfee8dcf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:49.947413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40733c72-7492-499e-a7e6-23cfee8dcf0f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:49.947695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40733c72-7492-499e-a7e6-23cfee8dcf0f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:49.947969 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40733c72-7492-499e-a7e6-23cfee8dcf0f None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:49.948742 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40733c72-7492-499e-a7e6-23cfee8dcf0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:49.949039 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40733c72-7492-499e-a7e6-23cfee8dcf0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:49.950458 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40733c72-7492-499e-a7e6-23cfee8dcf0f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:49.967413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 379/747] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:49.967413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40733c72-7492-499e-a7e6-23cfee8dcf0f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:50.037646 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:50.097274 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62f48e-305d-4d33-901b-a6d229cf0f1d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:50.139869 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:50.149338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818306ad-b9d0-403d-ba94-2182a9d8b997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:50.149879 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818306ad-b9d0-403d-ba94-2182a9d8b997 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:50.150158 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818306ad-b9d0-403d-ba94-2182a9d8b997 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:50.150431 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818306ad-b9d0-403d-ba94-2182a9d8b997 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:50.151079 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818306ad-b9d0-403d-ba94-2182a9d8b997 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:50.151394 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818306ad-b9d0-403d-ba94-2182a9d8b997 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:50.151507 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40733c72-7492-499e-a7e6-23cfee8dcf0f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:50.152831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818306ad-b9d0-403d-ba94-2182a9d8b997 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:50.168437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 369/748] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:50.169431 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818306ad-b9d0-403d-ba94-2182a9d8b997 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:50.206322 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b5482-8e91-4d51-aaca-68b3eff78371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:50.207679 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b5482-8e91-4d51-aaca-68b3eff78371 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:50.208673 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b5482-8e91-4d51-aaca-68b3eff78371 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:50.209150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b5482-8e91-4d51-aaca-68b3eff78371 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:50.221541 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b5482-8e91-4d51-aaca-68b3eff78371 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:50.223349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b5482-8e91-4d51-aaca-68b3eff78371 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:50.240326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-701b5482-8e91-4d51-aaca-68b3eff78371 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:42:50.240326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:50.240326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:50.240326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:50.240326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:50.240326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:50.240326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:50.240326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:50.240326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:50.240326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:50.240326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:50.240326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:50.240326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:50.240326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:50.240326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:50.240326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:50.242179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:50.242179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:50.242179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:50.242179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:50.242179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:50.242179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:50.242179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:50.242179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:50.242179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:50.242179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:50.242179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:50.242179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:50.242179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:50.242179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:50.242179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:50.242179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:50.242179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:50.242179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:50.243140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:50.243140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:50.243140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:50.243140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:50.243140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:50.243140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:50.243140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:50.243140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:50.243140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:42:50.243140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:50.246089 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 380/749] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 24 08:42:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:50.247103 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b5482-8e91-4d51-aaca-68b3eff78371 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:50.314741 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818306ad-b9d0-403d-ba94-2182a9d8b997 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:50.348295 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:50.365393 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f4e81-89c9-4619-98ec-1e677ab1a2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:50.366875 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f4e81-89c9-4619-98ec-1e677ab1a2ef None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:50.367629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f4e81-89c9-4619-98ec-1e677ab1a2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:50.367629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f4e81-89c9-4619-98ec-1e677ab1a2ef None admin] PATH_INFO: `/v3/users/nova` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:50.378919 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f4e81-89c9-4619-98ec-1e677ab1a2ef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:50.381180 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f4e81-89c9-4619-98ec-1e677ab1a2ef None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:50.395387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-431f4e81-89c9-4619-98ec-1e677ab1a2ef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:42:50.395387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:50.395387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:50.395387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:50.395387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:50.395387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:50.395387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:50.395387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:50.395387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:50.395387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:50.395387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:50.395387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:50.395387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:50.395387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:50.395387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:50.395387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:50.396497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:50.396497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:50.396497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:50.396497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:50.396497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:50.396497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:50.396497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:50.396497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:50.396497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:50.396497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:50.396497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:50.396497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:50.396497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:50.396497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:50.396497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:50.396497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:50.396497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:50.396497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:50.398088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:50.398088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:50.398088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:50.398088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:50.398088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:50.398088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:50.398088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:50.398088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:50.398088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:42:50.398088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:50.400630 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b5482-8e91-4d51-aaca-68b3eff78371 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:50.403521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 370/750] 10.222.0.67 () {56 vars in 1093 bytes} [Wed Aug 24 08:42:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:50.405145 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f4e81-89c9-4619-98ec-1e677ab1a2ef None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:50.453361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb08a1-d3e1-47fd-9438-367877b489e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:50.454429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb08a1-d3e1-47fd-9438-367877b489e0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:50.454884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb08a1-d3e1-47fd-9438-367877b489e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:50.455920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb08a1-d3e1-47fd-9438-367877b489e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:50.467130 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb08a1-d3e1-47fd-9438-367877b489e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:50.469781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb08a1-d3e1-47fd-9438-367877b489e0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:50.487645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-87bb08a1-d3e1-47fd-9438-367877b489e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:50.487645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:50.487645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:50.487645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:50.487645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:50.487645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:50.487645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:50.487645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:50.487645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:50.487645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:50.487645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:50.487645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:50.487645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:50.487645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:50.487645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:50.488810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:50.488810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:50.488810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:50.488810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:50.488810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:50.488810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:50.488810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:50.488810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:50.488810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:50.488810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:50.488810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:50.488810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:50.488810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:50.488810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:50.488810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:50.488810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:50.488810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:50.489720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:50.489720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:50.489720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:50.489720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:50.489720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:50.489720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:50.489720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:50.489720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:50.492767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 381/751] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:50.494052 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb08a1-d3e1-47fd-9438-367877b489e0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:50.514378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:50.550386 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f4e81-89c9-4619-98ec-1e677ab1a2ef None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:50.602917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79eb66-5944-4ad1-be65-f490ac401188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:50.603832 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79eb66-5944-4ad1-be65-f490ac401188 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:50.604307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79eb66-5944-4ad1-be65-f490ac401188 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:50.604760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79eb66-5944-4ad1-be65-f490ac401188 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:50.606398 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79eb66-5944-4ad1-be65-f490ac401188 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:50.607164 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79eb66-5944-4ad1-be65-f490ac401188 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:50.609510 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79eb66-5944-4ad1-be65-f490ac401188 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:50.619362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb08a1-d3e1-47fd-9438-367877b489e0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:50.639323 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 371/752] 10.222.0.67 () {56 vars in 1149 bytes} [Wed Aug 24 08:42:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:50.641252 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79eb66-5944-4ad1-be65-f490ac401188 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:50.675673 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca98145-1dd5-4c72-a43b-499b91c464cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:50.677054 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca98145-1dd5-4c72-a43b-499b91c464cd None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:50.677486 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca98145-1dd5-4c72-a43b-499b91c464cd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:50.677882 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca98145-1dd5-4c72-a43b-499b91c464cd None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:50.679737 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca98145-1dd5-4c72-a43b-499b91c464cd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:50.680130 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca98145-1dd5-4c72-a43b-499b91c464cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:50.683387 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca98145-1dd5-4c72-a43b-499b91c464cd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:50.702539 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:50.710910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 382/753] 10.222.0.67 () {56 vars in 1133 bytes} [Wed Aug 24 08:42:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:50.711960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca98145-1dd5-4c72-a43b-499b91c464cd None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:50.781861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79eb66-5944-4ad1-be65-f490ac401188 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:50.822592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f36703-132a-4707-adb5-48c82898f053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:50.823598 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f36703-132a-4707-adb5-48c82898f053 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:50.824056 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f36703-132a-4707-adb5-48c82898f053 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:50.824544 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f36703-132a-4707-adb5-48c82898f053 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:50.825545 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f36703-132a-4707-adb5-48c82898f053 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:50.826178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f36703-132a-4707-adb5-48c82898f053 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:50.827948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f36703-132a-4707-adb5-48c82898f053 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:50.844147 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:50.845744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 372/754] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:50.846991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f36703-132a-4707-adb5-48c82898f053 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:50.889565 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca98145-1dd5-4c72-a43b-499b91c464cd None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:50.938788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a87033-2717-4f91-8911-4785d2430522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:50.939897 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a87033-2717-4f91-8911-4785d2430522 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:50.940522 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a87033-2717-4f91-8911-4785d2430522 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:50.941211 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a87033-2717-4f91-8911-4785d2430522 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:50.992886 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f36703-132a-4707-adb5-48c82898f053 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:50.996657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:51.055807 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1041e4-c582-4289-aff2-a80cce19ca92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:51.056498 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1041e4-c582-4289-aff2-a80cce19ca92 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:51.056787 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1041e4-c582-4289-aff2-a80cce19ca92 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:51.057052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1041e4-c582-4289-aff2-a80cce19ca92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:51.069704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a87033-2717-4f91-8911-4785d2430522 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:51.071348 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a87033-2717-4f91-8911-4785d2430522 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:51.078181 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1041e4-c582-4289-aff2-a80cce19ca92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:51.082513 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1041e4-c582-4289-aff2-a80cce19ca92 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:51.126327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-7a1041e4-c582-4289-aff2-a80cce19ca92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:51.126327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:51.126327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:51.126327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:51.126327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:51.126327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:51.126327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:51.126327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:51.126327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:51.126327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:51.126327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:51.126327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:51.126327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:51.126327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:51.126327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:51.126327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:51.128239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:51.128239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:51.128239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:51.128239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:51.128239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:51.128239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:51.128239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:51.128239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:51.128239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:51.128239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:51.128239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:51.128239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:51.128239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:51.128239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:51.128239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:51.128239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:51.128239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:51.138894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:51.138894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:51.138894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:51.138894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 373/755] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:51.138894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1041e4-c582-4289-aff2-a80cce19ca92 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:51.173684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:51.220289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 383/756] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:50 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:42:51.220289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a87033-2717-4f91-8911-4785d2430522 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:51.315847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1041e4-c582-4289-aff2-a80cce19ca92 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:51.358411 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57ad19-1e87-4add-bbb5-ef6a8a93c6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:51.359610 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57ad19-1e87-4add-bbb5-ef6a8a93c6ba None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:51.360302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57ad19-1e87-4add-bbb5-ef6a8a93c6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:51.360740 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57ad19-1e87-4add-bbb5-ef6a8a93c6ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:51.362028 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57ad19-1e87-4add-bbb5-ef6a8a93c6ba None admin] RBAC: Adding query filter params (role.id=b13ffa72344c4856901b1ba003e273fd, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=46b1417d4ac24c8982b2052f4e99042a) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:51.362706 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57ad19-1e87-4add-bbb5-ef6a8a93c6ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:51.364596 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57ad19-1e87-4add-bbb5-ef6a8a93c6ba None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:51.380681 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 374/757] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:42:51 2022] GET /identity/v3/role_assignments?user.id=46b1417d4ac24c8982b2052f4e99042a&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=b13ffa72344c4856901b1ba003e273fd => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:51.382366 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57ad19-1e87-4add-bbb5-ef6a8a93c6ba None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:51.389689 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a87033-2717-4f91-8911-4785d2430522 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:51.443930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35a678-9963-4612-8aef-f5742fd31274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:51.447164 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:51.447487 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35a678-9963-4612-8aef-f5742fd31274 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:51.447766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35a678-9963-4612-8aef-f5742fd31274 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:51.450239 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35a678-9963-4612-8aef-f5742fd31274 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:51.451485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35a678-9963-4612-8aef-f5742fd31274 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:51.451998 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35a678-9963-4612-8aef-f5742fd31274 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:51.453833 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35a678-9963-4612-8aef-f5742fd31274 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:51.477497 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 384/758] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:51.494201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35a678-9963-4612-8aef-f5742fd31274 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:51.549124 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:51.562481 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57ad19-1e87-4add-bbb5-ef6a8a93c6ba None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:51.606992 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803d6bb-6309-40a8-84a2-e9d0bc644c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:51.606992 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803d6bb-6309-40a8-84a2-e9d0bc644c7a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:51.606992 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803d6bb-6309-40a8-84a2-e9d0bc644c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:51.607524 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803d6bb-6309-40a8-84a2-e9d0bc644c7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:51.619843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803d6bb-6309-40a8-84a2-e9d0bc644c7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:51.621368 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803d6bb-6309-40a8-84a2-e9d0bc644c7a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:51.637082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-a803d6bb-6309-40a8-84a2-e9d0bc644c7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:51.637082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:51.637082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:51.637082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:51.637082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:51.637082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:51.637082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:51.637082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:51.637082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:51.637082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:51.637082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:51.637082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:51.637082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:51.637082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:51.637082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:51.637082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:51.638934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:51.638934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:51.638934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:51.638934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:51.638934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:51.638934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:51.638934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:51.638934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:51.638934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:51.638934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:51.638934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:51.638934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:51.638934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:51.638934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:51.638934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:51.638934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:51.638934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:51.640759 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:51.640759 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:51.640759 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:51.640759 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 375/759] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:51.644615 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784f5e4-459d-476d-9199-beef66e503bf None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:51.644955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784f5e4-459d-476d-9199-beef66e503bf None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:51.645204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784f5e4-459d-476d-9199-beef66e503bf None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:51.646812 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 376/760] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:51.648275 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784f5e4-459d-476d-9199-beef66e503bf None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:51.697733 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35a678-9963-4612-8aef-f5742fd31274 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:51.752600 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:51.755922 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e1090-f42b-4c9b-8f65-6aa7822bb9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:51.757085 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e1090-f42b-4c9b-8f65-6aa7822bb9d8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:51.758207 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e1090-f42b-4c9b-8f65-6aa7822bb9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:51.758207 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e1090-f42b-4c9b-8f65-6aa7822bb9d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:51.765863 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e1090-f42b-4c9b-8f65-6aa7822bb9d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:51.767471 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e1090-f42b-4c9b-8f65-6aa7822bb9d8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:51.781376 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784f5e4-459d-476d-9199-beef66e503bf None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:51.790310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-307e1090-f42b-4c9b-8f65-6aa7822bb9d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:51.790310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:51.790310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:51.790310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:51.790310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:51.790310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:51.790310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:51.790310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:51.790310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:51.790310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:51.790310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:51.790310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:51.790310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:51.790310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:51.790310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:51.792332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:51.792332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:51.792332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:51.792332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:51.792332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:51.792332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:51.792332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:51.792332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:51.792332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:51.792332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:51.792332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:51.792332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:51.792332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:51.792332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:51.792332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:51.792332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:51.792332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:51.794217 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:51.794217 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:51.794217 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:51.794217 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:51.794217 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:51.794217 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:51.794217 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:51.794217 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:51.797638 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 385/761] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:51.800241 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79f540-6f23-498f-8220-848c11f138c8 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:51.801242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79f540-6f23-498f-8220-848c11f138c8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:51.801917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79f540-6f23-498f-8220-848c11f138c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:51.828942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61094e9d-dfc9-455e-96d0-4af7445bc5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:51.829569 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61094e9d-dfc9-455e-96d0-4af7445bc5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:51.829774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61094e9d-dfc9-455e-96d0-4af7445bc5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:51.830079 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61094e9d-dfc9-455e-96d0-4af7445bc5a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:51.830644 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61094e9d-dfc9-455e-96d0-4af7445bc5a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:51.830948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61094e9d-dfc9-455e-96d0-4af7445bc5a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:51.832370 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61094e9d-dfc9-455e-96d0-4af7445bc5a8 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:51.837420 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2f79f540-6f23-498f-8220-848c11f138c8 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:51.839823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2f79f540-6f23-498f-8220-848c11f138c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:51.853020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 377/762] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:51.854236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61094e9d-dfc9-455e-96d0-4af7445bc5a8 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:51.955343 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 386/763] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:42:51.957426 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79f540-6f23-498f-8220-848c11f138c8 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:52.002573 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61094e9d-dfc9-455e-96d0-4af7445bc5a8 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:52.016542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:52.044894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:52.052629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2276f-324f-4e75-b2e4-8e8c7db9ae88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:52.055096 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2276f-324f-4e75-b2e4-8e8c7db9ae88 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:52.055579 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2276f-324f-4e75-b2e4-8e8c7db9ae88 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:52.055988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2276f-324f-4e75-b2e4-8e8c7db9ae88 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:52.056871 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2276f-324f-4e75-b2e4-8e8c7db9ae88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:52.057314 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2276f-324f-4e75-b2e4-8e8c7db9ae88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:52.058901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2276f-324f-4e75-b2e4-8e8c7db9ae88 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:52.074762 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 378/764] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:52.083310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2276f-324f-4e75-b2e4-8e8c7db9ae88 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:52.167316 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79f540-6f23-498f-8220-848c11f138c8 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:52.216383 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72753e1-2699-461b-80e7-2e67e6a9aa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:52.217823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72753e1-2699-461b-80e7-2e67e6a9aa4f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:52.218553 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72753e1-2699-461b-80e7-2e67e6a9aa4f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:52.219003 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72753e1-2699-461b-80e7-2e67e6a9aa4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:52.229213 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2276f-324f-4e75-b2e4-8e8c7db9ae88 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:52.229602 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72753e1-2699-461b-80e7-2e67e6a9aa4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:52.231803 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72753e1-2699-461b-80e7-2e67e6a9aa4f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:52.248195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d72753e1-2699-461b-80e7-2e67e6a9aa4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:52.248195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:52.248195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:52.248195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:52.248195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:52.248195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:52.248195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:52.248195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:52.248195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:52.248195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:52.248195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:52.248195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:52.248195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:52.248195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:52.248195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:52.249162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:52.249162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:52.249162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:52.249162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:52.249162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:52.249162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:52.249162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:52.249162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:52.249162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:52.249162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:52.249162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:52.249162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:52.249162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:52.249162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:52.249162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:52.249162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:52.249162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:52.250045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:52.250045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:52.250045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:52.250045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:52.250045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:52.250045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:52.250045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:52.250045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:52.250045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 387/765] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:52.254481 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069c9a1-6f0a-400f-b313-0e89f2cfd945 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:52.254481 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069c9a1-6f0a-400f-b313-0e89f2cfd945 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:52.254481 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069c9a1-6f0a-400f-b313-0e89f2cfd945 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:52.254481 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 388/766] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:52.255846 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069c9a1-6f0a-400f-b313-0e89f2cfd945 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:52.284776 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11dc4a-df06-4277-abbe-b8fd2bd98e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:52.286195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11dc4a-df06-4277-abbe-b8fd2bd98e06 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:52.286687 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11dc4a-df06-4277-abbe-b8fd2bd98e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:52.287714 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11dc4a-df06-4277-abbe-b8fd2bd98e06 None admin] PATH_INFO: `/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/ca56dd7216fb4946946b2801e144fec2/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:52.292877 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11dc4a-df06-4277-abbe-b8fd2bd98e06 None admin] RBAC: Authorizing `identity:create_grant(project_id=fce6ee19f58b4a468c9a5d169e29a018, user_id=ca56dd7216fb4946946b2801e144fec2, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:52.296128 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11dc4a-df06-4277-abbe-b8fd2bd98e06 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:52.317279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 379/767] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:42:52 2022] PUT /identity/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/ca56dd7216fb4946946b2801e144fec2/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:42:52.320057 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ace9f2-a720-4b70-b0f5-906435b440fb None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:52.322211 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ace9f2-a720-4b70-b0f5-906435b440fb None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:52.322899 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ace9f2-a720-4b70-b0f5-906435b440fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:52.345076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-51ace9f2-a720-4b70-b0f5-906435b440fb None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:52.347235 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-51ace9f2-a720-4b70-b0f5-906435b440fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:52.416666 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:52.423261 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:52.447026 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069c9a1-6f0a-400f-b313-0e89f2cfd945 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:52.463274 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 380/768] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:42:52.472150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ace9f2-a720-4b70-b0f5-906435b440fb None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:52.498150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5d37f-6ed0-4a19-ac7d-6d925237d00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:52.498780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5d37f-6ed0-4a19-ac7d-6d925237d00d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:52.499060 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5d37f-6ed0-4a19-ac7d-6d925237d00d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:52.499335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5d37f-6ed0-4a19-ac7d-6d925237d00d None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:52.500053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab5d37f-6ed0-4a19-ac7d-6d925237d00d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:52.500385 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab5d37f-6ed0-4a19-ac7d-6d925237d00d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:52.501874 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab5d37f-6ed0-4a19-ac7d-6d925237d00d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:52.528416 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 389/769] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:52.537216 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5d37f-6ed0-4a19-ac7d-6d925237d00d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:52.641966 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ace9f2-a720-4b70-b0f5-906435b440fb None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:52.700346 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401405f6-fe26-4c4c-9125-be70214420f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:52.701350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401405f6-fe26-4c4c-9125-be70214420f6 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:52.702091 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401405f6-fe26-4c4c-9125-be70214420f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:52.702562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401405f6-fe26-4c4c-9125-be70214420f6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:52.703763 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401405f6-fe26-4c4c-9125-be70214420f6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:52.705697 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401405f6-fe26-4c4c-9125-be70214420f6 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:52.707454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5d37f-6ed0-4a19-ac7d-6d925237d00d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:52.717747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-401405f6-fe26-4c4c-9125-be70214420f6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:42:52.717747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:52.717747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:42:52.717747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:42:52.717747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:42:52.717747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:42:52.717747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:42:52.717747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:42:52.717747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:42:52.717747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:52.717747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:42:52.717747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:52.717747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:52.717747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:52.717747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:52.717747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:52.717747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:52.718779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:52.718779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:52.718779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:52.718779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:52.718779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:52.718779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:52.718779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:42:52.718779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:42:52.718779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:42:52.718779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:42:52.718779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:52.718779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:52.718779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:52.718779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:52.718779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:52.718779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:52.719631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:52.719631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:52.719631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:52.719631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:52.719631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:52.719631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:52.719631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:52.719631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:52.719631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:52.719631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:52.719631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:42:52.719631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:42:52.719631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:42:52.719631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:52.727907 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 381/770] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:52 2022] GET /identity/v3/services/network => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:52.731125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401405f6-fe26-4c4c-9125-be70214420f6 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:52.783968 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce744a7-6adf-4203-bb12-f85a6e006254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:52.784575 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce744a7-6adf-4203-bb12-f85a6e006254 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:52.784859 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce744a7-6adf-4203-bb12-f85a6e006254 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:52.785240 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce744a7-6adf-4203-bb12-f85a6e006254 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:52.785982 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce744a7-6adf-4203-bb12-f85a6e006254 None admin] RBAC: Adding query filter params (role.id=9aa748c8ea2542e6987e0b93190ea4e6, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=b11abed8d4424a1e9957c3dc9aefadd5) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:52.786294 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce744a7-6adf-4203-bb12-f85a6e006254 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:52.788340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce744a7-6adf-4203-bb12-f85a6e006254 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:52.800795 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 390/771] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:42:52 2022] GET /identity/v3/role_assignments?user.id=b11abed8d4424a1e9957c3dc9aefadd5&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=9aa748c8ea2542e6987e0b93190ea4e6 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:52.802242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce744a7-6adf-4203-bb12-f85a6e006254 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:52.817015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:52.831096 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:52.955397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401405f6-fe26-4c4c-9125-be70214420f6 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:53.027187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007c93c-6d64-4c16-871d-6412a26d5bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:53.027884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007c93c-6d64-4c16-871d-6412a26d5bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:53.028193 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007c93c-6d64-4c16-871d-6412a26d5bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:53.028413 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007c93c-6d64-4c16-871d-6412a26d5bf7 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:53.029084 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e007c93c-6d64-4c16-871d-6412a26d5bf7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:53.029391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e007c93c-6d64-4c16-871d-6412a26d5bf7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:53.030427 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce744a7-6adf-4203-bb12-f85a6e006254 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:53.031713 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e007c93c-6d64-4c16-871d-6412a26d5bf7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:53.041661 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 382/772] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 24 08:42:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:53.050171 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007c93c-6d64-4c16-871d-6412a26d5bf7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:53.117235 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f3142-8a0b-4077-bee9-6fbe440f99fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:53.117894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f3142-8a0b-4077-bee9-6fbe440f99fa None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:53.118168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f3142-8a0b-4077-bee9-6fbe440f99fa None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:53.118510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f3142-8a0b-4077-bee9-6fbe440f99fa None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:53.119299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f3142-8a0b-4077-bee9-6fbe440f99fa None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:53.121065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f3142-8a0b-4077-bee9-6fbe440f99fa None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:53.136600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-c66f3142-8a0b-4077-bee9-6fbe440f99fa None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:42:53.136600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:53.136600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:42:53.136600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:42:53.136600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:42:53.136600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:42:53.136600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:42:53.136600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:42:53.136600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:42:53.136600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:53.136600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:42:53.136600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:53.136600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:53.136600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:53.136600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:53.136600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:53.136600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:53.137712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:53.137712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:53.137712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:53.137712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:53.137712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:53.137712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:53.137712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:42:53.137712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:42:53.137712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:42:53.137712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:42:53.137712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:53.137712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:53.137712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:53.137712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:53.137712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:53.137712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:53.139030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:53.139030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:53.139030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:53.139030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:53.139030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:53.139030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:53.139030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:53.139030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:53.139030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:53.139030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:53.139030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:42:53.139030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:42:53.139030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:42:53.139030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:53.139030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 391/773] 10.222.0.67 () {56 vars in 1129 bytes} [Wed Aug 24 08:42:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:53.150867 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f3142-8a0b-4077-bee9-6fbe440f99fa None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:53.275330 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:53.308918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f3142-8a0b-4077-bee9-6fbe440f99fa None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:53.343403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007c93c-6d64-4c16-871d-6412a26d5bf7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:53.348497 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e99468-32f8-4dff-8fcf-0ca7d2a95fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:53.349117 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e99468-32f8-4dff-8fcf-0ca7d2a95fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:53.349393 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e99468-32f8-4dff-8fcf-0ca7d2a95fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:53.349670 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e99468-32f8-4dff-8fcf-0ca7d2a95fc5 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:53.350317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e99468-32f8-4dff-8fcf-0ca7d2a95fc5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:53.350731 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e99468-32f8-4dff-8fcf-0ca7d2a95fc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:53.352445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e99468-32f8-4dff-8fcf-0ca7d2a95fc5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:53.358903 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 392/774] 10.222.0.67 () {56 vars in 1124 bytes} [Wed Aug 24 08:42:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:53.368550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e99468-32f8-4dff-8fcf-0ca7d2a95fc5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:53.372549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:53.416645 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2699b4-6cf5-48e6-9804-f6393dc952a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:53.417493 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2699b4-6cf5-48e6-9804-f6393dc952a0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:53.418221 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2699b4-6cf5-48e6-9804-f6393dc952a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:53.418636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2699b4-6cf5-48e6-9804-f6393dc952a0 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:53.419800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2699b4-6cf5-48e6-9804-f6393dc952a0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:53.420136 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2699b4-6cf5-48e6-9804-f6393dc952a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:53.422146 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2699b4-6cf5-48e6-9804-f6393dc952a0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:53.429984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 383/775] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 24 08:42:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:53.437210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2699b4-6cf5-48e6-9804-f6393dc952a0 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:53.552655 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e99468-32f8-4dff-8fcf-0ca7d2a95fc5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:53.592872 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80440ebb-3cad-44ff-b260-07f22998a8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:53.594085 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80440ebb-3cad-44ff-b260-07f22998a8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:53.594627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80440ebb-3cad-44ff-b260-07f22998a8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:53.595068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80440ebb-3cad-44ff-b260-07f22998a8b5 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:53.595960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80440ebb-3cad-44ff-b260-07f22998a8b5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:53.596430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80440ebb-3cad-44ff-b260-07f22998a8b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:53.598028 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80440ebb-3cad-44ff-b260-07f22998a8b5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:53.600023 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:53.612847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 393/776] 10.222.0.67 () {56 vars in 1124 bytes} [Wed Aug 24 08:42:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:53.614535 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4575b3-641b-47bd-bded-98e167083db9 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:53.614972 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4575b3-641b-47bd-bded-98e167083db9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:53.615394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4575b3-641b-47bd-bded-98e167083db9 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:53.615732 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2699b4-6cf5-48e6-9804-f6393dc952a0 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:53.618289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 394/777] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:53.619452 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4575b3-641b-47bd-bded-98e167083db9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:53.656693 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5911b-13a0-4381-9efd-9f70963d0800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:53.657542 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5911b-13a0-4381-9efd-9f70963d0800 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:53.658004 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5911b-13a0-4381-9efd-9f70963d0800 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:53.658366 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5911b-13a0-4381-9efd-9f70963d0800 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:53.659360 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5911b-13a0-4381-9efd-9f70963d0800 None admin] RBAC: Adding query filter params (interface=public, service_id=41fe25a507294fd78466f6018afee61b, region_id=RegionOne) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:53.659761 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5911b-13a0-4381-9efd-9f70963d0800 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:53.661429 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5911b-13a0-4381-9efd-9f70963d0800 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:53.671059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 384/778] 10.222.0.67 () {56 vars in 1251 bytes} [Wed Aug 24 08:42:53 2022] GET /identity/v3/endpoints?service_id=41fe25a507294fd78466f6018afee61b&interface=public®ion_id=RegionOne => generated 194 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:53.677307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bf409-bcde-43f9-b7ca-15e058427d6f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:53.677673 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bf409-bcde-43f9-b7ca-15e058427d6f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:53.678115 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bf409-bcde-43f9-b7ca-15e058427d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:53.699364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-0c4bf409-bcde-43f9-b7ca-15e058427d6f None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:53.700930 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0c4bf409-bcde-43f9-b7ca-15e058427d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:53.735240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:53.741438 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4575b3-641b-47bd-bded-98e167083db9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:53.784319 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 385/779] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:42:53.784319 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4bf409-bcde-43f9-b7ca-15e058427d6f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:53.792495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59ec51-970a-4db5-aac1-5f31e4aada82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:53.793676 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59ec51-970a-4db5-aac1-5f31e4aada82 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:53.794349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59ec51-970a-4db5-aac1-5f31e4aada82 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:53.794891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59ec51-970a-4db5-aac1-5f31e4aada82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:53.795863 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e59ec51-970a-4db5-aac1-5f31e4aada82 None admin] RBAC: Adding query filter params (interface=public, service_id=b39489ac05c646728360276a045ff156, region_id=RegionOne) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:53.796492 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e59ec51-970a-4db5-aac1-5f31e4aada82 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:53.798311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e59ec51-970a-4db5-aac1-5f31e4aada82 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:53.815812 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 395/780] 10.222.0.67 () {56 vars in 1251 bytes} [Wed Aug 24 08:42:53 2022] GET /identity/v3/endpoints?service_id=b39489ac05c646728360276a045ff156&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:53.821803 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59ec51-970a-4db5-aac1-5f31e4aada82 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:53.922238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4bf409-bcde-43f9-b7ca-15e058427d6f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:53.968487 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:53.996486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d2931-f291-49c9-9141-6e8a41a11cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:53.997106 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d2931-f291-49c9-9141-6e8a41a11cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:53.997386 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d2931-f291-49c9-9141-6e8a41a11cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:53.997659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d2931-f291-49c9-9141-6e8a41a11cb4 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:53.998271 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d2931-f291-49c9-9141-6e8a41a11cb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:53.999638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d2931-f291-49c9-9141-6e8a41a11cb4 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:54.010142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 386/781] 10.222.0.67 () {56 vars in 1087 bytes} [Wed Aug 24 08:42:53 2022] GET /identity/v3/services => generated 772 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:54.012911 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59ec51-970a-4db5-aac1-5f31e4aada82 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:54.066066 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3770be-56e4-459a-b4c1-fa67521caf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:54.067153 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3770be-56e4-459a-b4c1-fa67521caf01 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:54.067640 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3770be-56e4-459a-b4c1-fa67521caf01 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:54.068212 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3770be-56e4-459a-b4c1-fa67521caf01 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:54.069058 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3770be-56e4-459a-b4c1-fa67521caf01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:54.070669 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3770be-56e4-459a-b4c1-fa67521caf01 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:54.077031 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 396/782] 10.222.0.67 () {56 vars in 1087 bytes} [Wed Aug 24 08:42:53 2022] GET /identity/v3/services => generated 772 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:42:54.123066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d2931-f291-49c9-9141-6e8a41a11cb4 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:54.160752 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:54.245129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d2931-f291-49c9-9141-6e8a41a11cb4 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:54.285970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7386b7-da10-4f07-a023-e40353b56e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:54.286625 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7386b7-da10-4f07-a023-e40353b56e51 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:54.286907 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7386b7-da10-4f07-a023-e40353b56e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:54.287180 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7386b7-da10-4f07-a023-e40353b56e51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:54.300454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7386b7-da10-4f07-a023-e40353b56e51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:54.305829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-9f7386b7-da10-4f07-a023-e40353b56e51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:42:54.305829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:54.305829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:54.305829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:54.305829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:54.305829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:54.305829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:54.305829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:54.305829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:54.305829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:54.305829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:54.305829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:54.305829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:54.305829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:54.305829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:54.305829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:54.306862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:54.306862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:54.306862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:54.306862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:54.306862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:54.306862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:54.306862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:54.306862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:54.306862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:54.306862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:54.306862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:54.306862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:54.306862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:54.306862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:54.306862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:54.306862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:54.307698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:54.307698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:54.307698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:54.307698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:54.307698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:54.307698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:54.307698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:54.307698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:54.307698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:54.307698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:54.307698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:42:54.307698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:54.307698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 387/783] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:42:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:54.313569 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3770be-56e4-459a-b4c1-fa67521caf01 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:54.439012 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3770be-56e4-459a-b4c1-fa67521caf01 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:54.474297 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:54.489737 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087bc7a-b810-4191-9df2-13e25bc1cd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:54.490421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087bc7a-b810-4191-9df2-13e25bc1cd8c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:54.490704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087bc7a-b810-4191-9df2-13e25bc1cd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:54.491211 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087bc7a-b810-4191-9df2-13e25bc1cd8c None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:54.491897 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087bc7a-b810-4191-9df2-13e25bc1cd8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:54.492229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087bc7a-b810-4191-9df2-13e25bc1cd8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:54.493607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087bc7a-b810-4191-9df2-13e25bc1cd8c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:54.509858 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 397/784] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:42:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:54.515459 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7386b7-da10-4f07-a023-e40353b56e51 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:54.643948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:54.656255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7386b7-da10-4f07-a023-e40353b56e51 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:54.708029 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bce733-be82-48f4-a8dc-c63d73526923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:54.708728 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bce733-be82-48f4-a8dc-c63d73526923 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:54.709017 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bce733-be82-48f4-a8dc-c63d73526923 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:54.709290 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bce733-be82-48f4-a8dc-c63d73526923 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:54.733495 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bce733-be82-48f4-a8dc-c63d73526923 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:54.735035 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bce733-be82-48f4-a8dc-c63d73526923 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:54.767089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-a5bce733-be82-48f4-a8dc-c63d73526923 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:54.767089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:54.767089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:54.767089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:54.767089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:54.767089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:54.767089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:54.767089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:54.767089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:54.767089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:54.767089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:54.767089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:54.767089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:54.767089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:54.767089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:54.767089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:54.768066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:54.768066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:54.768066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:54.768066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:54.768066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:54.768066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:54.768066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:54.768066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:54.768066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:54.768066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:54.768066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:54.768066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:54.768066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:54.768066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:54.768066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:54.768066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:54.768066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:54.769001 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:54.769001 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:54.769001 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:54.769001 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 388/785] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:54.776940 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087bc7a-b810-4191-9df2-13e25bc1cd8c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:54.782484 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bffc5-8f28-4447-89ac-2ee8e85dd05f None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:54.782798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bffc5-8f28-4447-89ac-2ee8e85dd05f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:54.783045 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bffc5-8f28-4447-89ac-2ee8e85dd05f None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:54.789381 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 389/786] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:54.797815 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84a54f-b3da-4239-bc14-487a9d5ae923 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:54.798372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84a54f-b3da-4239-bc14-487a9d5ae923 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:54.798656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84a54f-b3da-4239-bc14-487a9d5ae923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:54.842923 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9c84a54f-b3da-4239-bc14-487a9d5ae923 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:54.844550 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9c84a54f-b3da-4239-bc14-487a9d5ae923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:54.966869 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087bc7a-b810-4191-9df2-13e25bc1cd8c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:55.020378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d088c2-7870-4c79-a2aa-bcec914ba528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:55.020922 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d088c2-7870-4c79-a2aa-bcec914ba528 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:55.021199 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d088c2-7870-4c79-a2aa-bcec914ba528 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:55.021469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d088c2-7870-4c79-a2aa-bcec914ba528 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:55.022134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d088c2-7870-4c79-a2aa-bcec914ba528 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:55.022600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d088c2-7870-4c79-a2aa-bcec914ba528 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:55.024203 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d088c2-7870-4c79-a2aa-bcec914ba528 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:55.030501 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 390/787] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:42:55.046926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 398/788] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:55.052780 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84a54f-b3da-4239-bc14-487a9d5ae923 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:55.186533 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:55.242590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84a54f-b3da-4239-bc14-487a9d5ae923 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:55.291225 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0875836-3071-4c72-9872-c3d733f872a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:55.292203 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0875836-3071-4c72-9872-c3d733f872a4 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:55.292669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0875836-3071-4c72-9872-c3d733f872a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:55.293122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0875836-3071-4c72-9872-c3d733f872a4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:55.303222 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0875836-3071-4c72-9872-c3d733f872a4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:55.305633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0875836-3071-4c72-9872-c3d733f872a4 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:55.329294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-f0875836-3071-4c72-9872-c3d733f872a4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:42:55.329294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:55.329294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:55.329294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:55.329294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:55.329294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:55.329294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:55.329294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:55.329294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:55.329294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:55.329294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:55.329294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:55.329294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:55.329294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:55.329294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:55.329294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:55.330301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:55.330301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:55.330301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:55.330301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:55.330301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:55.330301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:55.330301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:55.330301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:55.330301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:55.330301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:55.330301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:55.330301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:55.330301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:55.330301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:55.330301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:55.330301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:55.330301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:55.330301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:55.331224 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:55.331224 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:55.331224 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:55.331224 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:55.331224 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:55.331224 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:55.331224 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:55.331224 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:55.331224 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:42:55.331224 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:55.332289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 391/789] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:42:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:55.339409 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d088c2-7870-4c79-a2aa-bcec914ba528 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:55.417021 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0875836-3071-4c72-9872-c3d733f872a4 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:55.419831 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:55.530446 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d088c2-7870-4c79-a2aa-bcec914ba528 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:55.589396 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc3818-33fe-43c4-ae9e-f0b18c6934fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:55.589918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc3818-33fe-43c4-ae9e-f0b18c6934fd None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:55.590193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc3818-33fe-43c4-ae9e-f0b18c6934fd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:55.590482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc3818-33fe-43c4-ae9e-f0b18c6934fd None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:55.591720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc3818-33fe-43c4-ae9e-f0b18c6934fd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:55.592044 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc3818-33fe-43c4-ae9e-f0b18c6934fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:55.593876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc3818-33fe-43c4-ae9e-f0b18c6934fd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:55.595712 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0875836-3071-4c72-9872-c3d733f872a4 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:55.621154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 399/790] 10.222.0.67 () {56 vars in 1135 bytes} [Wed Aug 24 08:42:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:55.622578 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5224649-fe9d-4684-bacf-d00e048bda08 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:55.622872 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5224649-fe9d-4684-bacf-d00e048bda08 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:55.623156 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5224649-fe9d-4684-bacf-d00e048bda08 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:55.624790 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 400/791] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:55.634037 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5224649-fe9d-4684-bacf-d00e048bda08 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:55.647011 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3808a-afe0-476f-9238-c509416cafc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:55.648735 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3808a-afe0-476f-9238-c509416cafc2 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:55.649071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3808a-afe0-476f-9238-c509416cafc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:55.649343 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3808a-afe0-476f-9238-c509416cafc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:55.652440 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:55.667497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3808a-afe0-476f-9238-c509416cafc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:55.674000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-64b3808a-afe0-476f-9238-c509416cafc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:42:55.674000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:55.674000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:55.674000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:55.674000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:55.674000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:55.674000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:55.674000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:55.674000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:55.674000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:55.674000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:55.674000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:55.674000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:55.674000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:55.674000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:55.674000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:55.675236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:55.675236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:55.675236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:55.675236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:55.675236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:55.675236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:55.675236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:55.675236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:55.675236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:55.675236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:55.675236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:55.675236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:55.675236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:55.675236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:55.675236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:55.675236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:55.676502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:55.676502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:55.676502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:55.676502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:55.676502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:55.676502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:55.676502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:55.676502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:55.676502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:55.676502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:55.676502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:42:55.676502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:55.676502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 392/792] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:42:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:55.677211 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8524bf-d5c1-455b-8d50-53020d437d96 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:55.677288 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8524bf-d5c1-455b-8d50-53020d437d96 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:55.677524 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8524bf-d5c1-455b-8d50-53020d437d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:55.702368 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-0f8524bf-d5c1-455b-8d50-53020d437d96 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:55.703558 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0f8524bf-d5c1-455b-8d50-53020d437d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:55.803956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5224649-fe9d-4684-bacf-d00e048bda08 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:55.813862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:55.820473 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 393/793] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:42:55.822356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8524bf-d5c1-455b-8d50-53020d437d96 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:55.862470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7642a85-0274-40ec-96c1-255e57d85f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:55.863078 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7642a85-0274-40ec-96c1-255e57d85f14 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:55.863353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7642a85-0274-40ec-96c1-255e57d85f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:55.864028 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7642a85-0274-40ec-96c1-255e57d85f14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:55.888971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7642a85-0274-40ec-96c1-255e57d85f14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:55.890432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7642a85-0274-40ec-96c1-255e57d85f14 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:55.933167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d7642a85-0274-40ec-96c1-255e57d85f14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:55.933167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:55.933167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:55.933167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:55.933167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:55.933167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:55.933167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:55.933167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:55.933167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:55.933167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:55.933167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:55.933167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:55.933167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:55.933167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:55.933167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:55.933167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:55.934941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:55.934941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:55.934941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:55.934941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:55.934941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:55.934941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:55.934941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:55.934941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:55.934941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:55.934941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:55.934941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:55.934941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:55.934941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:55.934941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:55.934941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:55.934941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:55.934941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:55.936716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:55.936716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:55.936716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:55.936716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 401/794] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:55.949166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7642a85-0274-40ec-96c1-255e57d85f14 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:55.991075 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8524bf-d5c1-455b-8d50-53020d437d96 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:56.020862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:56.043253 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73654be0-09d2-449e-8043-7c03635ef587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:56.043975 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73654be0-09d2-449e-8043-7c03635ef587 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:56.044284 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73654be0-09d2-449e-8043-7c03635ef587 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:56.044582 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73654be0-09d2-449e-8043-7c03635ef587 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:56.045222 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73654be0-09d2-449e-8043-7c03635ef587 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:56.045527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73654be0-09d2-449e-8043-7c03635ef587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:56.047159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73654be0-09d2-449e-8043-7c03635ef587 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:56.071254 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 394/795] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:42:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:56.079778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73654be0-09d2-449e-8043-7c03635ef587 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:56.113348 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7642a85-0274-40ec-96c1-255e57d85f14 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:56.181515 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4bb08-9a11-41d0-b047-c5bd3745d20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:56.182138 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4bb08-9a11-41d0-b047-c5bd3745d20c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:56.182402 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4bb08-9a11-41d0-b047-c5bd3745d20c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:56.182671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4bb08-9a11-41d0-b047-c5bd3745d20c None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:56.183321 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4bb08-9a11-41d0-b047-c5bd3745d20c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:56.183623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4bb08-9a11-41d0-b047-c5bd3745d20c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:56.185028 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4bb08-9a11-41d0-b047-c5bd3745d20c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:56.200234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 402/796] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:56.209370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4bb08-9a11-41d0-b047-c5bd3745d20c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:56.246859 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:56.290750 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73654be0-09d2-449e-8043-7c03635ef587 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:56.342825 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8aa33-98be-47a2-9d40-8612c7d2f81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:56.343895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8aa33-98be-47a2-9d40-8612c7d2f81d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:56.344958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8aa33-98be-47a2-9d40-8612c7d2f81d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:56.345444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8aa33-98be-47a2-9d40-8612c7d2f81d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:56.359596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4bb08-9a11-41d0-b047-c5bd3745d20c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:56.362991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8aa33-98be-47a2-9d40-8612c7d2f81d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:56.364965 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8aa33-98be-47a2-9d40-8612c7d2f81d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:56.393214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-02e8aa33-98be-47a2-9d40-8612c7d2f81d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:56.393214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:56.393214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:56.393214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:56.393214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:56.393214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:56.393214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:56.393214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:56.393214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:56.393214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:56.393214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:56.393214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:56.393214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:56.393214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:56.393214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:56.393214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:56.395111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:56.395111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:56.395111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:56.395111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:56.395111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:56.395111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:56.395111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:56.395111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:56.395111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:56.395111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:56.395111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:56.395111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:56.395111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:56.395111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:56.395111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:56.395111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:56.395111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:56.396963 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:56.396963 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:56.396963 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:56.407086 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 395/797] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:42:56.409232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8aa33-98be-47a2-9d40-8612c7d2f81d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:56.492881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:56.526853 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51914a47-7da1-4103-a641-93d09cf4fd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:56.527599 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51914a47-7da1-4103-a641-93d09cf4fd33 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:56.527881 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51914a47-7da1-4103-a641-93d09cf4fd33 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:56.528173 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51914a47-7da1-4103-a641-93d09cf4fd33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:56.535629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51914a47-7da1-4103-a641-93d09cf4fd33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:56.537297 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51914a47-7da1-4103-a641-93d09cf4fd33 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:56.549967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-51914a47-7da1-4103-a641-93d09cf4fd33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:56.549967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:56.549967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:56.549967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:56.549967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:56.549967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:56.549967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:56.549967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:56.549967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:56.549967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:56.549967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:56.549967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:56.549967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:56.549967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:56.549967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:56.551931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:56.551931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:56.551931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:56.551931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:56.551931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:56.551931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:56.551931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:56.551931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:56.551931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:56.551931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:56.551931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:56.551931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:56.551931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:56.551931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:56.551931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:56.551931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:56.551931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:56.554269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:56.554269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:56.554269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:56.554269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:56.554269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:56.554269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:56.554269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:56.554269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:56.554269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 403/798] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:42:56.554269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51914a47-7da1-4103-a641-93d09cf4fd33 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:56.606514 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8aa33-98be-47a2-9d40-8612c7d2f81d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:56.648526 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa9064-a960-4aa3-9bcf-93ce0beeddfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:56.649196 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa9064-a960-4aa3-9bcf-93ce0beeddfb None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:56.649542 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa9064-a960-4aa3-9bcf-93ce0beeddfb None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:56.649793 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa9064-a960-4aa3-9bcf-93ce0beeddfb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:56.660546 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:56.664755 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa9064-a960-4aa3-9bcf-93ce0beeddfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:42:56.670463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-cffa9064-a960-4aa3-9bcf-93ce0beeddfb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:56.670463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:56.670463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:56.670463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:56.670463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:56.670463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:56.670463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:56.670463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:56.670463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:56.670463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:56.670463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:56.670463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:56.670463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:42:56.670463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:42:56.670463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:42:56.670463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:42:56.672529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:42:56.672529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:42:56.672529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:42:56.672529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:42:56.672529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:56.672529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:56.672529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:56.672529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:56.672529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:56.672529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:56.672529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:56.672529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:56.672529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:56.672529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:56.672529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:56.672529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:56.674311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:56.674311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:56.674311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:56.674311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:56.674311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:42:56.674311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:42:56.674311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:42:56.674311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:42:56.674311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:42:56.674311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:42:56.674311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:42:56.674311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:56.674311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 396/799] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:42:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:56.674311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eca6fc-2966-43ef-91cb-6053630e2dfb None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:56.675810 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eca6fc-2966-43ef-91cb-6053630e2dfb None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:56.675810 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eca6fc-2966-43ef-91cb-6053630e2dfb None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:56.680059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 397/800] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:56.681066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eca6fc-2966-43ef-91cb-6053630e2dfb None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:56.695581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51914a47-7da1-4103-a641-93d09cf4fd33 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:56.732725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebbf4a-d88d-4e14-a459-dd8f95ea4adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:56.733640 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebbf4a-d88d-4e14-a459-dd8f95ea4adb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:56.733920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebbf4a-d88d-4e14-a459-dd8f95ea4adb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:56.734191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebbf4a-d88d-4e14-a459-dd8f95ea4adb None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:56.734843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdebbf4a-d88d-4e14-a459-dd8f95ea4adb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:56.735145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdebbf4a-d88d-4e14-a459-dd8f95ea4adb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:56.736565 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdebbf4a-d88d-4e14-a459-dd8f95ea4adb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:56.748348 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 404/801] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:56.749596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fd821-bf55-4e52-9aca-b229a7773319 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:56.749882 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fd821-bf55-4e52-9aca-b229a7773319 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:56.750153 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fd821-bf55-4e52-9aca-b229a7773319 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:56.751505 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 405/802] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:42:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:42:56.752031 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84fd821-bf55-4e52-9aca-b229a7773319 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:56.794123 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:56.799258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eca6fc-2966-43ef-91cb-6053630e2dfb None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:56.841837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75b156-99ff-4766-b1cd-e9c0476c768f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:56.844438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75b156-99ff-4766-b1cd-e9c0476c768f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:56.844438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75b156-99ff-4766-b1cd-e9c0476c768f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:56.844438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75b156-99ff-4766-b1cd-e9c0476c768f None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:56.844438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e75b156-99ff-4766-b1cd-e9c0476c768f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:56.844438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e75b156-99ff-4766-b1cd-e9c0476c768f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:56.845453 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e75b156-99ff-4766-b1cd-e9c0476c768f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:56.858787 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 398/803] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:56.860030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63c05b-6165-4e5e-a90d-e53b12cf22ec None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:56.860345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63c05b-6165-4e5e-a90d-e53b12cf22ec None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:56.860612 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63c05b-6165-4e5e-a90d-e53b12cf22ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:56.865640 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84fd821-bf55-4e52-9aca-b229a7773319 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:56.879146 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-fe63c05b-6165-4e5e-a90d-e53b12cf22ec None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:56.881229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fe63c05b-6165-4e5e-a90d-e53b12cf22ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:56.906273 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e2ce5-ab4e-4a5a-bec6-b339bb19fe67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:56.906804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e2ce5-ab4e-4a5a-bec6-b339bb19fe67 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:56.907118 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e2ce5-ab4e-4a5a-bec6-b339bb19fe67 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:56.907394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e2ce5-ab4e-4a5a-bec6-b339bb19fe67 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:56.908118 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e2ce5-ab4e-4a5a-bec6-b339bb19fe67 None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:56.908426 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e2ce5-ab4e-4a5a-bec6-b339bb19fe67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:56.909806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e2ce5-ab4e-4a5a-bec6-b339bb19fe67 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:56.915371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:56.919538 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 406/804] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:42:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:56.920595 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e2ce5-ab4e-4a5a-bec6-b339bb19fe67 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:56.959955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 399/805] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:42:56.961793 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4be1a8-21a4-4217-b02e-0ac550d89bf2 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:56.962035 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4be1a8-21a4-4217-b02e-0ac550d89bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:56.962304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4be1a8-21a4-4217-b02e-0ac550d89bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:56.980492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-2e4be1a8-21a4-4217-b02e-0ac550d89bf2 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:42:56.981675 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2e4be1a8-21a4-4217-b02e-0ac550d89bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:42:57.041682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e2ce5-ab4e-4a5a-bec6-b339bb19fe67 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:57.076814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 400/806] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:42:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:42:57.077643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4be1a8-21a4-4217-b02e-0ac550d89bf2 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:57.094386 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d268932-863a-45b7-9c87-6ba6f1e3e20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:57.095090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d268932-863a-45b7-9c87-6ba6f1e3e20b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:57.095376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d268932-863a-45b7-9c87-6ba6f1e3e20b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:57.095644 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d268932-863a-45b7-9c87-6ba6f1e3e20b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:57.105446 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d268932-863a-45b7-9c87-6ba6f1e3e20b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:57.106523 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d268932-863a-45b7-9c87-6ba6f1e3e20b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:57.122877 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:57.129966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-5d268932-863a-45b7-9c87-6ba6f1e3e20b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:42:57.129966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:57.129966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:57.129966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:57.129966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:57.129966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:57.129966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:57.129966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:57.129966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:57.129966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:57.129966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:57.129966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:57.129966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:57.129966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:57.129966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:57.129966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:57.132362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:57.132362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:57.132362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:57.132362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:57.132362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:57.132362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:57.132362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:57.132362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:57.132362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:57.132362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:57.132362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:57.132362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:57.132362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:57.132362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:57.132362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:57.132362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:57.132362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:57.132362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:57.134123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:57.134123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:57.134123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:57.134123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:57.134123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:57.134123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:57.134123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:57.134123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:57.134123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:42:57.134123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:57.137675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 407/807] 10.222.0.67 () {56 vars in 1093 bytes} [Wed Aug 24 08:42:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:57.139618 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d268932-863a-45b7-9c87-6ba6f1e3e20b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:57.239451 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:57.259575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4be1a8-21a4-4217-b02e-0ac550d89bf2 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:57.296785 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d268932-863a-45b7-9c87-6ba6f1e3e20b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:57.305047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a17f7a-8898-441b-ac68-902efa636e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:57.306367 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a17f7a-8898-441b-ac68-902efa636e9c None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:57.307405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a17f7a-8898-441b-ac68-902efa636e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:57.307841 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a17f7a-8898-441b-ac68-902efa636e9c None admin] PATH_INFO: `/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/46b1417d4ac24c8982b2052f4e99042a/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:57.310807 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a17f7a-8898-441b-ac68-902efa636e9c None admin] RBAC: Authorizing `identity:create_grant(project_id=fce6ee19f58b4a468c9a5d169e29a018, user_id=46b1417d4ac24c8982b2052f4e99042a, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:57.312876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a17f7a-8898-441b-ac68-902efa636e9c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:57.332199 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 401/808] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:42:57 2022] PUT /identity/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/46b1417d4ac24c8982b2052f4e99042a/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:42:57.333930 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a17f7a-8898-441b-ac68-902efa636e9c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:57.344485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b8e0b-5b3b-4772-8c23-c52df08e96c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:57.345223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b8e0b-5b3b-4772-8c23-c52df08e96c3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:57.345570 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b8e0b-5b3b-4772-8c23-c52df08e96c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:57.345929 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b8e0b-5b3b-4772-8c23-c52df08e96c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:57.359030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b8e0b-5b3b-4772-8c23-c52df08e96c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:57.360660 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b8e0b-5b3b-4772-8c23-c52df08e96c3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:57.377646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-b57b8e0b-5b3b-4772-8c23-c52df08e96c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:57.377646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:57.377646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:57.377646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:57.377646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:57.377646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:57.377646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:57.377646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:57.377646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:57.377646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:57.377646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:57.377646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:57.377646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:57.377646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:57.377646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:57.377646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:57.378678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:57.378678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:57.378678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:57.378678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:57.378678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:57.378678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:57.378678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:57.378678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:57.378678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:57.378678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:57.378678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:57.378678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:57.378678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:57.378678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:57.378678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:57.378678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:57.378678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:57.379597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:57.379597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:57.379597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:57.384931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 408/809] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:57.385898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b8e0b-5b3b-4772-8c23-c52df08e96c3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:57.464239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:57.494396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a17f7a-8898-441b-ac68-902efa636e9c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:57.540748 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958dff0-9f4b-4847-961f-afe050322b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:57.541720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958dff0-9f4b-4847-961f-afe050322b83 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:57.542007 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958dff0-9f4b-4847-961f-afe050322b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:57.542276 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958dff0-9f4b-4847-961f-afe050322b83 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:57.543050 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958dff0-9f4b-4847-961f-afe050322b83 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:57.543491 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958dff0-9f4b-4847-961f-afe050322b83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:57.545232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958dff0-9f4b-4847-961f-afe050322b83 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:57.567970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 402/810] 10.222.0.67 () {56 vars in 1133 bytes} [Wed Aug 24 08:42:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:57.568901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958dff0-9f4b-4847-961f-afe050322b83 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:57.581964 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b8e0b-5b3b-4772-8c23-c52df08e96c3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:57.608386 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:57.637014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df0977-aa5f-43f4-b120-1269c98ec344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:57.637668 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df0977-aa5f-43f4-b120-1269c98ec344 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:57.637945 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df0977-aa5f-43f4-b120-1269c98ec344 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:57.638219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df0977-aa5f-43f4-b120-1269c98ec344 None admin] PATH_INFO: `/v3/services/network` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:57.638825 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df0977-aa5f-43f4-b120-1269c98ec344 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:57.640350 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df0977-aa5f-43f4-b120-1269c98ec344 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:57.652325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d0df0977-aa5f-43f4-b120-1269c98ec344 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:42:57.652325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:57.652325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:42:57.652325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:42:57.652325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:42:57.652325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:42:57.652325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:42:57.652325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:42:57.652325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:42:57.652325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:57.652325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:42:57.652325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:57.652325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:57.652325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:57.652325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:57.652325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:57.652325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:57.653475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:57.653475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:57.653475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:57.653475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:57.653475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:57.653475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:57.653475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:42:57.653475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:42:57.653475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:42:57.653475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:42:57.653475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:57.653475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:57.653475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:57.653475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:57.653475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:57.653475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:57.654743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:57.654743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:57.654743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:57.654743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:57.654743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:57.654743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:57.654743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:57.654743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:57.654743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:57.654743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:57.654743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:42:57.654743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:42:57.654743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 08:42:57.654743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:57.654743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 409/811] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:57 2022] GET /identity/v3/services/network => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:57.655969 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df0977-aa5f-43f4-b120-1269c98ec344 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:57.753575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958dff0-9f4b-4847-961f-afe050322b83 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:57.789184 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df0977-aa5f-43f4-b120-1269c98ec344 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:57.808290 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a3f8c-2e80-42a5-a50a-7d8dc69c14e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:57.809086 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a3f8c-2e80-42a5-a50a-7d8dc69c14e7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:57.809546 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a3f8c-2e80-42a5-a50a-7d8dc69c14e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:57.809976 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a3f8c-2e80-42a5-a50a-7d8dc69c14e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:57.810956 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a3f8c-2e80-42a5-a50a-7d8dc69c14e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:57.811379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a3f8c-2e80-42a5-a50a-7d8dc69c14e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:57.812965 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a3f8c-2e80-42a5-a50a-7d8dc69c14e7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:57.816428 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:57.836929 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 403/812] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:57.838941 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a3f8c-2e80-42a5-a50a-7d8dc69c14e7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:57.840171 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbcfc2-5789-45db-98ac-9186e749401e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:57.841126 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbcfc2-5789-45db-98ac-9186e749401e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:57.841581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbcfc2-5789-45db-98ac-9186e749401e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:57.842022 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbcfc2-5789-45db-98ac-9186e749401e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:57.842983 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bbcfc2-5789-45db-98ac-9186e749401e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:57.845294 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bbcfc2-5789-45db-98ac-9186e749401e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:57.860615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-b8bbcfc2-5789-45db-98ac-9186e749401e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:42:57.860615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:57.860615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:42:57.860615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:42:57.860615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:42:57.860615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:42:57.860615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:42:57.860615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:42:57.860615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:42:57.860615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:57.860615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:42:57.860615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:57.860615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:57.860615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:57.860615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:57.860615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:57.860615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:57.861687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:57.861687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:57.861687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:57.861687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:57.861687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:57.861687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:57.861687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:42:57.861687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:42:57.861687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:42:57.861687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:42:57.861687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:57.861687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:57.861687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:57.861687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:57.861687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:57.861687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:57.862526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:57.862526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:57.862526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:57.862526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:57.862526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:57.862526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:57.862526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:57.862526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:57.862526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:57.862526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:57.862526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:42:57.862526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:42:57.862526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 08:42:57.862526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:57.867904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 410/813] 10.222.0.67 () {56 vars in 1129 bytes} [Wed Aug 24 08:42:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:57.868901 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbcfc2-5789-45db-98ac-9186e749401e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:57.928349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:57.998387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a3f8c-2e80-42a5-a50a-7d8dc69c14e7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:58.028920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbcfc2-5789-45db-98ac-9186e749401e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:58.047447 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25c857-f7f5-4096-9a90-57ebcbd84923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:58.048088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25c857-f7f5-4096-9a90-57ebcbd84923 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:58.048414 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25c857-f7f5-4096-9a90-57ebcbd84923 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:58.048665 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25c857-f7f5-4096-9a90-57ebcbd84923 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:58.072848 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30380e-52b7-4089-8019-3965ad190d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:58.073482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30380e-52b7-4089-8019-3965ad190d00 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:58.073761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30380e-52b7-4089-8019-3965ad190d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:58.074033 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30380e-52b7-4089-8019-3965ad190d00 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:58.074681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30380e-52b7-4089-8019-3965ad190d00 None admin] RBAC: Adding query filter params (name=network) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:58.074984 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30380e-52b7-4089-8019-3965ad190d00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:58.076406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30380e-52b7-4089-8019-3965ad190d00 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:58.086336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 411/814] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 24 08:42:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:58.086336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30380e-52b7-4089-8019-3965ad190d00 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:58.142839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:58.153711 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c25c857-f7f5-4096-9a90-57ebcbd84923 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:58.155887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c25c857-f7f5-4096-9a90-57ebcbd84923 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:58.229419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30380e-52b7-4089-8019-3965ad190d00 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:58.232306 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:58.270520 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f224e-25c6-44e7-a320-0924100d6b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:58.271657 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f224e-25c6-44e7-a320-0924100d6b33 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:58.272152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f224e-25c6-44e7-a320-0924100d6b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:58.273020 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f224e-25c6-44e7-a320-0924100d6b33 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:58.288934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 404/815] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:42:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:42:58.288934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25c857-f7f5-4096-9a90-57ebcbd84923 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:58.291033 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f224e-25c6-44e7-a320-0924100d6b33 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:58.292537 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f224e-25c6-44e7-a320-0924100d6b33 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:58.305149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-4c4f224e-25c6-44e7-a320-0924100d6b33 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:42:58.305149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:58.305149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:58.305149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:58.305149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:58.305149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:58.305149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:58.305149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:58.305149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:58.305149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:58.305149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:58.305149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:58.305149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:42:58.305149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:42:58.305149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:42:58.305149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:42:58.306247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:58.306247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:58.306247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:42:58.306247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:58.306247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:42:58.306247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:42:58.306247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:58.306247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:58.306247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:58.306247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:58.306247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:58.306247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:58.306247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:58.306247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:58.306247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:58.306247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:58.306247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:58.306247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:58.307271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:58.307271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:58.307271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:42:58.307271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:42:58.307271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:42:58.307271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:42:58.307271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:42:58.307271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:42:58.307271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:42:58.307271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:58.307271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 412/816] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 24 08:42:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:58.307271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f224e-25c6-44e7-a320-0924100d6b33 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:58.420919 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:58.432948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25c857-f7f5-4096-9a90-57ebcbd84923 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:58.449280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f224e-25c6-44e7-a320-0924100d6b33 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:58.495572 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcf23e-ebd2-4563-9717-e10b493304a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:58.496580 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcf23e-ebd2-4563-9717-e10b493304a3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:58.497157 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcf23e-ebd2-4563-9717-e10b493304a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:58.497607 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcf23e-ebd2-4563-9717-e10b493304a3 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:58.498567 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcf23e-ebd2-4563-9717-e10b493304a3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:58.499090 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcf23e-ebd2-4563-9717-e10b493304a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:58.500810 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcf23e-ebd2-4563-9717-e10b493304a3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:58.505133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7f408-754c-4316-94f3-260293e7d83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:58.505927 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7f408-754c-4316-94f3-260293e7d83c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:58.506344 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7f408-754c-4316-94f3-260293e7d83c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:58.507359 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7f408-754c-4316-94f3-260293e7d83c None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:58.508194 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7f408-754c-4316-94f3-260293e7d83c None admin] RBAC: Adding query filter params (type=network) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:58.508647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7f408-754c-4316-94f3-260293e7d83c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:58.516028 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 405/817] 10.222.0.67 () {56 vars in 1124 bytes} [Wed Aug 24 08:42:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:58.516298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7f408-754c-4316-94f3-260293e7d83c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:58.516432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcf23e-ebd2-4563-9717-e10b493304a3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:58.521768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 413/818] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 24 08:42:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:58.523816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7f408-754c-4316-94f3-260293e7d83c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:58.558986 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:58.652254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7f408-754c-4316-94f3-260293e7d83c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:58.696643 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a8f6c-a1ba-4b9e-83d9-4f7f228c98f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:58.697270 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a8f6c-a1ba-4b9e-83d9-4f7f228c98f3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:58.697572 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a8f6c-a1ba-4b9e-83d9-4f7f228c98f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:58.697819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a8f6c-a1ba-4b9e-83d9-4f7f228c98f3 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:58.698567 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a8f6c-a1ba-4b9e-83d9-4f7f228c98f3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:58.698870 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a8f6c-a1ba-4b9e-83d9-4f7f228c98f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:58.700470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a8f6c-a1ba-4b9e-83d9-4f7f228c98f3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:58.727307 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 414/819] 10.222.0.67 () {56 vars in 1149 bytes} [Wed Aug 24 08:42:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:58.729337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a8f6c-a1ba-4b9e-83d9-4f7f228c98f3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:58.864418 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcf23e-ebd2-4563-9717-e10b493304a3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:58.869978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:58.886337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a8f6c-a1ba-4b9e-83d9-4f7f228c98f3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:58.919296 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d83dd-49fa-45c8-b1f9-5250009728ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:58.920381 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d83dd-49fa-45c8-b1f9-5250009728ac None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:58.920846 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d83dd-49fa-45c8-b1f9-5250009728ac None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:58.921318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d83dd-49fa-45c8-b1f9-5250009728ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:58.924454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b2037-05cb-40ae-80ba-562598755da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:58.925773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b2037-05cb-40ae-80ba-562598755da6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:58.927227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b2037-05cb-40ae-80ba-562598755da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:58.927498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b2037-05cb-40ae-80ba-562598755da6 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:58.928172 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b2037-05cb-40ae-80ba-562598755da6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:58.928523 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b2037-05cb-40ae-80ba-562598755da6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:58.930296 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b2037-05cb-40ae-80ba-562598755da6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:58.940688 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 415/820] 10.222.0.67 () {56 vars in 1124 bytes} [Wed Aug 24 08:42:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:58.940688 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b2037-05cb-40ae-80ba-562598755da6 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:58.945861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d83dd-49fa-45c8-b1f9-5250009728ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:58.948455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d83dd-49fa-45c8-b1f9-5250009728ac None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:58.972798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-926d83dd-49fa-45c8-b1f9-5250009728ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:58.972798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:58.972798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:58.972798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:58.972798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:58.972798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:58.972798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:58.972798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:58.972798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:58.972798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:58.972798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:58.972798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:58.972798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:58.972798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:58.972798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:58.972798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:58.973835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:58.973835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:58.973835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:58.973835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:58.973835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:58.973835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:58.973835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:58.973835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:58.973835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:58.973835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:58.973835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:58.973835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:58.973835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:58.973835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:58.973835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:58.973835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:58.973835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:58.974722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:58.974722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:58.974722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:58.982631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 406/821] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 472 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:58.983707 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d83dd-49fa-45c8-b1f9-5250009728ac None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:59.010246 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:59.095046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b2037-05cb-40ae-80ba-562598755da6 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:59.117331 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d83dd-49fa-45c8-b1f9-5250009728ac None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:59.144146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d3ecf-cbb5-46f1-8f15-7ff91aec5506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:59.144773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d3ecf-cbb5-46f1-8f15-7ff91aec5506 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:59.145061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d3ecf-cbb5-46f1-8f15-7ff91aec5506 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:59.145332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d3ecf-cbb5-46f1-8f15-7ff91aec5506 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:59.146013 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d3ecf-cbb5-46f1-8f15-7ff91aec5506 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:59.150386 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d3ecf-cbb5-46f1-8f15-7ff91aec5506 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:59.165214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab188dc-5406-42f8-a2c2-b6baa2938ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:59.166183 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab188dc-5406-42f8-a2c2-b6baa2938ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:59.166645 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab188dc-5406-42f8-a2c2-b6baa2938ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:59.167102 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab188dc-5406-42f8-a2c2-b6baa2938ba0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:59.183372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab188dc-5406-42f8-a2c2-b6baa2938ba0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:59.184939 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab188dc-5406-42f8-a2c2-b6baa2938ba0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:59.194237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 416/822] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:42:58 2022] POST /identity/v3/endpoints => generated 337 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:42:59.194237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d3ecf-cbb5-46f1-8f15-7ff91aec5506 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:59.203254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:59.220308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-2ab188dc-5406-42f8-a2c2-b6baa2938ba0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:59.220308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:59.220308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:59.220308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:59.220308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:59.220308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:59.220308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:59.220308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:59.220308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:59.220308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:59.220308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:59.220308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:59.220308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:42:59.220308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:42:59.220308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:42:59.220308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:42:59.222175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:59.222175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:59.222175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:59.222175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:59.222175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:59.222175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:59.222175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:59.222175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:59.222175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:59.222175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:59.222175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:59.222175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:59.222175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:59.222175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:59.222175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:59.222175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:59.222175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:42:59.223885 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:42:59.223885 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:42:59.223885 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:42:59.228542 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 407/823] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:42:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:59.230039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab188dc-5406-42f8-a2c2-b6baa2938ba0 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:59.356488 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:59.394899 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d3ecf-cbb5-46f1-8f15-7ff91aec5506 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:59.407343 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab188dc-5406-42f8-a2c2-b6baa2938ba0 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:59.443805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1f158-d970-4541-baac-f552290e8960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:59.444688 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1f158-d970-4541-baac-f552290e8960 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:59.445537 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1f158-d970-4541-baac-f552290e8960 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:59.445537 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1f158-d970-4541-baac-f552290e8960 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:59.445817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e1f158-d970-4541-baac-f552290e8960 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:59.447136 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e1f158-d970-4541-baac-f552290e8960 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:59.452444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5363fe-32d8-46a1-8a8b-13539a1d970a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:59.453280 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5363fe-32d8-46a1-8a8b-13539a1d970a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:59.453452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5363fe-32d8-46a1-8a8b-13539a1d970a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:59.453612 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5363fe-32d8-46a1-8a8b-13539a1d970a None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:59.454253 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5363fe-32d8-46a1-8a8b-13539a1d970a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:59.454565 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5363fe-32d8-46a1-8a8b-13539a1d970a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:59.455961 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5363fe-32d8-46a1-8a8b-13539a1d970a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:59.470958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 408/824] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:59.471861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5363fe-32d8-46a1-8a8b-13539a1d970a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:59.504770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 417/825] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:42:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:42:59.505744 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1f158-d970-4541-baac-f552290e8960 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:59.609015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:59.666459 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5363fe-32d8-46a1-8a8b-13539a1d970a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:59.698697 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:42:59.710093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1f158-d970-4541-baac-f552290e8960 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:59.719823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542ef52-7cc1-4b45-8be9-9d0fcffc0e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:59.719823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542ef52-7cc1-4b45-8be9-9d0fcffc0e86 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:59.719823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542ef52-7cc1-4b45-8be9-9d0fcffc0e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:59.719823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542ef52-7cc1-4b45-8be9-9d0fcffc0e86 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:59.720542 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542ef52-7cc1-4b45-8be9-9d0fcffc0e86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:42:59.720542 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542ef52-7cc1-4b45-8be9-9d0fcffc0e86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:59.721204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542ef52-7cc1-4b45-8be9-9d0fcffc0e86 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:59.746271 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 409/826] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:42:59.746517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542ef52-7cc1-4b45-8be9-9d0fcffc0e86 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:59.754891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b36a7-156d-4207-9023-6f107f8dddc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:59.755530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b36a7-156d-4207-9023-6f107f8dddc7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:59.755811 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b36a7-156d-4207-9023-6f107f8dddc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:59.756071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b36a7-156d-4207-9023-6f107f8dddc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:59.771253 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b36a7-156d-4207-9023-6f107f8dddc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:42:59.774876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b36a7-156d-4207-9023-6f107f8dddc7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:42:59.799264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-8d9b36a7-156d-4207-9023-6f107f8dddc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:59.799264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:42:59.799264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:42:59.799264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:42:59.799264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:42:59.799264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:42:59.799264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:42:59.799264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:42:59.799264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:42:59.799264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:42:59.799264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:42:59.799264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:42:59.799264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:42:59.799264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:42:59.799264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:42:59.800584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:42:59.800584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:42:59.800584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:42:59.800584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:42:59.800584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:42:59.800584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:42:59.800584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:42:59.800584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:42:59.800584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:42:59.800584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:42:59.800584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:42:59.800584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:42:59.800584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:42:59.800584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:42:59.800584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:42:59.800584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:42:59.800584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:42:59.801768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:42:59.801768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:42:59.801768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:42:59.801768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:42:59.801768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:42:59.801768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:42:59.801768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:42:59.801768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:42:59.802221 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 418/827] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:42:59.811612 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b36a7-156d-4207-9023-6f107f8dddc7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:42:59.917823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542ef52-7cc1-4b45-8be9-9d0fcffc0e86 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:59.983002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650bf8d-2df2-4e16-8de9-c113a1803706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:42:59.984313 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650bf8d-2df2-4e16-8de9-c113a1803706 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:42:59.984651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650bf8d-2df2-4e16-8de9-c113a1803706 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:42:59.985113 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650bf8d-2df2-4e16-8de9-c113a1803706 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:42:59.988035 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b36a7-156d-4207-9023-6f107f8dddc7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:42:59.998390 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650bf8d-2df2-4e16-8de9-c113a1803706 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:00.001298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650bf8d-2df2-4e16-8de9-c113a1803706 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:00.009761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:00.025059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-c650bf8d-2df2-4e16-8de9-c113a1803706 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:00.025059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:00.025059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:00.025059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:00.025059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:00.025059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:00.025059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:00.025059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:00.025059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:00.025059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:00.025059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:00.025059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:00.025059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:43:00.025059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:43:00.025059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:43:00.026200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:43:00.026200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:00.026200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:00.026200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:00.026200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:00.026200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:00.026200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:00.026200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:00.026200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:00.026200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:00.026200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:00.026200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:00.026200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:00.026200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:00.026200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:00.026200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:00.026200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:00.027240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:43:00.027240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:43:00.027240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:43:00.027240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:43:00.027240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:43:00.027240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:43:00.027240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:00.027240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:00.037426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 410/828] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:42:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:00.039611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9d010-ba7a-4e7d-a04a-163665565971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:00.040327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650bf8d-2df2-4e16-8de9-c113a1803706 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:00.040675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9d010-ba7a-4e7d-a04a-163665565971 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:00.041195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9d010-ba7a-4e7d-a04a-163665565971 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:00.041649 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9d010-ba7a-4e7d-a04a-163665565971 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:00.042546 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a9d010-ba7a-4e7d-a04a-163665565971 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:00.043034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a9d010-ba7a-4e7d-a04a-163665565971 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:00.044804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a9d010-ba7a-4e7d-a04a-163665565971 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:00.062308 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 419/829] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:42:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:00.071207 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9d010-ba7a-4e7d-a04a-163665565971 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:00.118876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:00.225284 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650bf8d-2df2-4e16-8de9-c113a1803706 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:00.241047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9d010-ba7a-4e7d-a04a-163665565971 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:00.278207 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5556ec-ea2b-4514-826c-4e899efc1d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:00.278846 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5556ec-ea2b-4514-826c-4e899efc1d12 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:00.279127 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5556ec-ea2b-4514-826c-4e899efc1d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:00.279401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5556ec-ea2b-4514-826c-4e899efc1d12 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:00.280156 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5556ec-ea2b-4514-826c-4e899efc1d12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:00.280470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5556ec-ea2b-4514-826c-4e899efc1d12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:00.282034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5556ec-ea2b-4514-826c-4e899efc1d12 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:00.299461 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 411/830] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:43:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:00.308888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5556ec-ea2b-4514-826c-4e899efc1d12 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:00.314455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328b0bd-dc03-4c2d-97c6-4d55f091c326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:00.315799 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328b0bd-dc03-4c2d-97c6-4d55f091c326 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:00.316528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328b0bd-dc03-4c2d-97c6-4d55f091c326 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:00.317003 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328b0bd-dc03-4c2d-97c6-4d55f091c326 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:00.318367 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328b0bd-dc03-4c2d-97c6-4d55f091c326 None admin] RBAC: Adding query filter params (role.id=b13ffa72344c4856901b1ba003e273fd, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=ca56dd7216fb4946946b2801e144fec2) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:00.318859 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328b0bd-dc03-4c2d-97c6-4d55f091c326 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:00.321284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328b0bd-dc03-4c2d-97c6-4d55f091c326 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:00.340579 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 420/831] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:43:00 2022] GET /identity/v3/role_assignments?user.id=ca56dd7216fb4946946b2801e144fec2&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=b13ffa72344c4856901b1ba003e273fd => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:00.495296 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a82d2-b25b-49f0-a082-4a89d444bac1 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:00.495296 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a82d2-b25b-49f0-a082-4a89d444bac1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:00.495660 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a82d2-b25b-49f0-a082-4a89d444bac1 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:00.497258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 421/832] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:00.500984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5556ec-ea2b-4514-826c-4e899efc1d12 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:00.503258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6584ba7-11ce-4712-b960-d8ae0af310dc None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:00.503681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6584ba7-11ce-4712-b960-d8ae0af310dc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:00.504051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6584ba7-11ce-4712-b960-d8ae0af310dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:00.507375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:00.527702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b6584ba7-11ce-4712-b960-d8ae0af310dc None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:00.529460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b6584ba7-11ce-4712-b960-d8ae0af310dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:00.568167 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae4ad6-16fd-4586-9b59-807886b609ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:00.568167 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae4ad6-16fd-4586-9b59-807886b609ca None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:00.568167 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae4ad6-16fd-4586-9b59-807886b609ca None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:00.568167 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae4ad6-16fd-4586-9b59-807886b609ca None admin] PATH_INFO: `/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/b11abed8d4424a1e9957c3dc9aefadd5/roles/9aa748c8ea2542e6987e0b93190ea4e6` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:00.579973 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:00.598203 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae4ad6-16fd-4586-9b59-807886b609ca None admin] RBAC: Authorizing `identity:create_grant(project_id=fce6ee19f58b4a468c9a5d169e29a018, user_id=b11abed8d4424a1e9957c3dc9aefadd5, role_id=9aa748c8ea2542e6987e0b93190ea4e6)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:00.599785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae4ad6-16fd-4586-9b59-807886b609ca None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:00.626886 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 412/833] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:43:00 2022] PUT /identity/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/b11abed8d4424a1e9957c3dc9aefadd5/roles/9aa748c8ea2542e6987e0b93190ea4e6 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:43:00.650164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 422/834] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:43:00.940299 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae4ad6-16fd-4586-9b59-807886b609ca None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:01.127535 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae4ad6-16fd-4586-9b59-807886b609ca None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:01.193002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591bd1eb-3e00-4fb2-b4da-8dd64017d7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:01.193907 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bd1eb-3e00-4fb2-b4da-8dd64017d7eb None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:01.194904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bd1eb-3e00-4fb2-b4da-8dd64017d7eb None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:01.195364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bd1eb-3e00-4fb2-b4da-8dd64017d7eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:01.214396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591bd1eb-3e00-4fb2-b4da-8dd64017d7eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:43:01.223637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-591bd1eb-3e00-4fb2-b4da-8dd64017d7eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:43:01.223637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:01.223637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:01.223637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:01.223637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:01.223637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:01.223637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:01.223637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:01.223637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:01.223637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:01.223637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:01.223637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:01.223637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:43:01.223637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:43:01.223637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:43:01.223637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:43:01.226012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:43:01.226012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:43:01.226012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:43:01.226012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:43:01.226012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:01.226012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:01.226012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:01.226012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:01.226012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:01.226012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:01.226012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:01.226012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:01.226012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:01.226012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:01.226012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:01.226012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:01.228861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:01.228861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:01.228861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:01.228861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:01.228861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:43:01.228861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:43:01.228861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:43:01.228861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:43:01.228861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:43:01.228861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:43:01.228861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:43:01.228861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:01.228861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 413/835] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:43:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:01.237927 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6584ba7-11ce-4712-b960-d8ae0af310dc None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:01.437135 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6584ba7-11ce-4712-b960-d8ae0af310dc None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:01.488849 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934414f-be2b-400a-96cd-90489ee22fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:01.489378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934414f-be2b-400a-96cd-90489ee22fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:01.489650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934414f-be2b-400a-96cd-90489ee22fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:01.489922 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934414f-be2b-400a-96cd-90489ee22fa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:01.490621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934414f-be2b-400a-96cd-90489ee22fa4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:01.490921 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934414f-be2b-400a-96cd-90489ee22fa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:01.493359 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934414f-be2b-400a-96cd-90489ee22fa4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:01.503719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 423/836] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:43:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:01.510058 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591bd1eb-3e00-4fb2-b4da-8dd64017d7eb None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:01.532006 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:01.636121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591bd1eb-3e00-4fb2-b4da-8dd64017d7eb None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:01.697520 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95863a-97a0-4ea1-a962-1ecba4683767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:01.698556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95863a-97a0-4ea1-a962-1ecba4683767 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:01.699033 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95863a-97a0-4ea1-a962-1ecba4683767 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:01.699497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95863a-97a0-4ea1-a962-1ecba4683767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:01.719383 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95863a-97a0-4ea1-a962-1ecba4683767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:01.722140 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95863a-97a0-4ea1-a962-1ecba4683767 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:01.752744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-9b95863a-97a0-4ea1-a962-1ecba4683767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:01.752744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:01.752744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:01.752744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:01.752744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:01.752744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:01.752744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:01.752744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:01.752744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:01.752744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:01.752744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:01.752744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:01.752744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:43:01.752744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:43:01.752744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:43:01.752744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:43:01.755892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:01.755892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:01.755892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:01.755892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:01.755892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:01.755892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:01.755892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:01.755892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:01.755892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:01.755892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:01.755892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:01.755892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:01.755892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:01.755892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:01.755892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:01.755892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:01.755892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:43:01.758799 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:43:01.758799 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:01.758799 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:01.758799 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 414/837] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:01.764122 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934414f-be2b-400a-96cd-90489ee22fa4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:01.804312 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:01.946116 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934414f-be2b-400a-96cd-90489ee22fa4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:02.012403 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298f46d-1c65-4861-9946-bc3aa3830b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:02.013360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298f46d-1c65-4861-9946-bc3aa3830b76 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:02.013951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298f46d-1c65-4861-9946-bc3aa3830b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:02.014459 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298f46d-1c65-4861-9946-bc3aa3830b76 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:02.015663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298f46d-1c65-4861-9946-bc3aa3830b76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:02.016248 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298f46d-1c65-4861-9946-bc3aa3830b76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:02.019123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298f46d-1c65-4861-9946-bc3aa3830b76 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:02.046020 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 424/838] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:02.051129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95863a-97a0-4ea1-a962-1ecba4683767 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:02.148410 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:02.215437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95863a-97a0-4ea1-a962-1ecba4683767 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:02.259336 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6556541-8086-49a8-8a96-26376b228095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:02.259984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6556541-8086-49a8-8a96-26376b228095 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:02.260275 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6556541-8086-49a8-8a96-26376b228095 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:02.260550 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6556541-8086-49a8-8a96-26376b228095 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:02.269332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6556541-8086-49a8-8a96-26376b228095 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:02.270697 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6556541-8086-49a8-8a96-26376b228095 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:02.284277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-c6556541-8086-49a8-8a96-26376b228095 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:43:02.284277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:02.284277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:02.284277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:02.284277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:02.284277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:02.284277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:02.284277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:02.284277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:02.284277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:02.284277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:02.284277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:02.284277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:43:02.284277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:43:02.284277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:43:02.284277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:43:02.287656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:02.287656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:02.287656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:43:02.287656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:02.287656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:43:02.287656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:02.287656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:02.287656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:02.287656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:02.287656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:02.287656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:02.287656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:02.287656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:02.287656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:02.287656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:02.287656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:02.287656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:02.287656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:02.291375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:02.291375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:02.291375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:43:02.291375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:43:02.291375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:43:02.291375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:43:02.291375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:43:02.291375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:43:02.291375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 08:43:02.291375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:02.291375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 415/839] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:43:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:02.297725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298f46d-1c65-4861-9946-bc3aa3830b76 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:02.404031 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:02.415215 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298f46d-1c65-4861-9946-bc3aa3830b76 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:02.456454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb814c5-fbaf-4f4a-878a-a27cbbf756e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:02.457092 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb814c5-fbaf-4f4a-878a-a27cbbf756e6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:02.457344 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb814c5-fbaf-4f4a-878a-a27cbbf756e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:02.457622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb814c5-fbaf-4f4a-878a-a27cbbf756e6 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:02.458385 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb814c5-fbaf-4f4a-878a-a27cbbf756e6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:02.458683 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb814c5-fbaf-4f4a-878a-a27cbbf756e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:02.460917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb814c5-fbaf-4f4a-878a-a27cbbf756e6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:02.481889 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 425/840] 10.222.0.67 () {56 vars in 1135 bytes} [Wed Aug 24 08:43:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:02.488442 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6556541-8086-49a8-8a96-26376b228095 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:02.634506 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6556541-8086-49a8-8a96-26376b228095 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:02.672688 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:02.676060 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae5ba4-5d85-49d2-b5ca-89a534195c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:02.677000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae5ba4-5d85-49d2-b5ca-89a534195c2f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:02.677619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae5ba4-5d85-49d2-b5ca-89a534195c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:02.677869 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae5ba4-5d85-49d2-b5ca-89a534195c2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:02.784182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae5ba4-5d85-49d2-b5ca-89a534195c2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:02.786996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae5ba4-5d85-49d2-b5ca-89a534195c2f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:02.875479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53487c6-8398-45df-9c72-140a326f8ffd None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:02.875976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53487c6-8398-45df-9c72-140a326f8ffd None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:02.876543 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53487c6-8398-45df-9c72-140a326f8ffd None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:02.884290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 426/841] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:02.889304 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cf576-3842-459d-997a-1758b70a015b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:02.889738 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cf576-3842-459d-997a-1758b70a015b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:02.890236 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cf576-3842-459d-997a-1758b70a015b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:02.906681 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 416/842] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:43:02 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 08:43:02.912177 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae5ba4-5d85-49d2-b5ca-89a534195c2f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:02.921863 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a25cf576-3842-459d-997a-1758b70a015b None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:02.923975 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a25cf576-3842-459d-997a-1758b70a015b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:02.980501 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:03.045551 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae5ba4-5d85-49d2-b5ca-89a534195c2f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:03.096380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ef8f1-dd17-4679-8f2f-372eb00c95a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:03.097439 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ef8f1-dd17-4679-8f2f-372eb00c95a0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:03.097916 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ef8f1-dd17-4679-8f2f-372eb00c95a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:03.098438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ef8f1-dd17-4679-8f2f-372eb00c95a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:03.101873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 427/843] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:43:03.107888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c38b45-eef0-40b7-abe8-14d8311bbc62 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:03.108204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c38b45-eef0-40b7-abe8-14d8311bbc62 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:03.108473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c38b45-eef0-40b7-abe8-14d8311bbc62 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:03.110135 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 428/844] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:03.120133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add75a20-1f88-4422-8c40-e4c74cf2ec97 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:03.120133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add75a20-1f88-4422-8c40-e4c74cf2ec97 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:03.120133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add75a20-1f88-4422-8c40-e4c74cf2ec97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:03.126249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ef8f1-dd17-4679-8f2f-372eb00c95a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:03.127709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ef8f1-dd17-4679-8f2f-372eb00c95a0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:03.139905 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-add75a20-1f88-4422-8c40-e4c74cf2ec97 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:03.141046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-add75a20-1f88-4422-8c40-e4c74cf2ec97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:03.171676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-d32ef8f1-dd17-4679-8f2f-372eb00c95a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:03.171676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:03.171676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:03.171676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:03.171676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:03.171676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:03.171676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:03.171676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:03.171676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:03.171676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:03.171676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:03.171676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:03.171676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:43:03.171676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:43:03.171676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:43:03.171676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:43:03.173590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:03.173590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:03.173590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:03.173590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:03.173590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:03.173590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:03.173590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:03.173590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:03.173590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:03.173590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:03.173590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:03.173590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:03.173590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:03.173590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:03.173590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:03.173590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:03.173590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:43:03.176896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:43:03.176896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:03.176896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:03.177794 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 417/845] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:43:03.180821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ef8f1-dd17-4679-8f2f-372eb00c95a0 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:03.252529 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 429/846] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:03 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:43:03.357006 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:03.391107 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add75a20-1f88-4422-8c40-e4c74cf2ec97 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:03.404355 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:03.405646 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ef8f1-dd17-4679-8f2f-372eb00c95a0 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:03.448721 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e3ec0-1dca-47fd-b7e6-4283d3a16495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:03.449278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e3ec0-1dca-47fd-b7e6-4283d3a16495 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:03.449558 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e3ec0-1dca-47fd-b7e6-4283d3a16495 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:03.449826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e3ec0-1dca-47fd-b7e6-4283d3a16495 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:03.450490 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e3ec0-1dca-47fd-b7e6-4283d3a16495 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:03.450764 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e3ec0-1dca-47fd-b7e6-4283d3a16495 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:03.452245 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e3ec0-1dca-47fd-b7e6-4283d3a16495 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:03.466660 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 418/847] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:03.475133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e3ec0-1dca-47fd-b7e6-4283d3a16495 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:03.542918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add75a20-1f88-4422-8c40-e4c74cf2ec97 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:03.590551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa25e4-2fda-46e3-a86f-3ec64066115d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:03.592458 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa25e4-2fda-46e3-a86f-3ec64066115d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:03.593049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa25e4-2fda-46e3-a86f-3ec64066115d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:03.594214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa25e4-2fda-46e3-a86f-3ec64066115d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:03.595110 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa25e4-2fda-46e3-a86f-3ec64066115d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:03.597596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa25e4-2fda-46e3-a86f-3ec64066115d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:03.612068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-2cfa25e4-2fda-46e3-a86f-3ec64066115d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:43:03.612068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:03.612068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:43:03.612068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:43:03.612068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:43:03.612068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:43:03.612068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:43:03.612068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:03.612068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:43:03.612068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:03.612068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:03.612068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:03.612068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:03.612068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:03.612068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:03.612068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:03.612068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:03.613432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:03.613432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:03.613432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:03.613432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:03.613432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:03.613432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:03.613432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:43:03.613432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:43:03.613432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:43:03.613432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:43:03.613432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:03.613432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:03.613432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:03.613432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:03.613432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:03.613432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:03.614284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:03.614284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:03.614284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:03.614284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:03.614284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:03.614284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:03.614284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:03.614284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:03.614284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:03.614284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:03.614284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:43:03.614284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:03.614284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:43:03.614284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:03.618050 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 430/848] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:43:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:03.627901 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa25e4-2fda-46e3-a86f-3ec64066115d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:03.668296 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e3ec0-1dca-47fd-b7e6-4283d3a16495 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:03.711039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0e5ba-9390-4bc7-87a9-06bd617603d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:03.711699 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0e5ba-9390-4bc7-87a9-06bd617603d6 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:03.711982 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0e5ba-9390-4bc7-87a9-06bd617603d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:03.712302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0e5ba-9390-4bc7-87a9-06bd617603d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:03.720319 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d0e5ba-9390-4bc7-87a9-06bd617603d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:03.722085 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d0e5ba-9390-4bc7-87a9-06bd617603d6 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:03.724384 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:03.737454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-19d0e5ba-9390-4bc7-87a9-06bd617603d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:03.737454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:03.737454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:03.737454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:03.737454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:03.737454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:03.737454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:03.737454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:03.737454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:03.737454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:03.737454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:03.737454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:03.737454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:43:03.737454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:43:03.737454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:43:03.738629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:43:03.738629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:03.738629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:03.738629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:03.738629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:03.738629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:03.738629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:03.738629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:03.738629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:03.738629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:03.738629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:03.738629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:03.738629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:03.738629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:03.738629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:03.738629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:03.738629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:03.739622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:43:03.739622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:43:03.739622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:43:03.739622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:43:03.739622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:43:03.739622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:43:03.739622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:03.739622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:03.739622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 419/849] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:43:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:03.740564 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0e5ba-9390-4bc7-87a9-06bd617603d6 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:03.746980 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:03.781714 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa25e4-2fda-46e3-a86f-3ec64066115d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:03.818577 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487f7ca-3263-4e10-89ab-3994d916455c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:03.820179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487f7ca-3263-4e10-89ab-3994d916455c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:03.820469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487f7ca-3263-4e10-89ab-3994d916455c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:03.820935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487f7ca-3263-4e10-89ab-3994d916455c None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:03.821589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487f7ca-3263-4e10-89ab-3994d916455c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:03.821890 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487f7ca-3263-4e10-89ab-3994d916455c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:03.823275 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487f7ca-3263-4e10-89ab-3994d916455c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:03.828680 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 431/850] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:43:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:03.829848 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e292d3-cd85-4eb8-b26a-6a2aa118f80e None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:03.830227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e292d3-cd85-4eb8-b26a-6a2aa118f80e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:03.830498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e292d3-cd85-4eb8-b26a-6a2aa118f80e None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:03.831952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 432/851] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:03.832573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e292d3-cd85-4eb8-b26a-6a2aa118f80e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:03.868517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0e5ba-9390-4bc7-87a9-06bd617603d6 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:03.907378 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221568bb-928e-4e05-99fc-8f4ec9c42892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:03.908127 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221568bb-928e-4e05-99fc-8f4ec9c42892 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:03.908403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221568bb-928e-4e05-99fc-8f4ec9c42892 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:03.908674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221568bb-928e-4e05-99fc-8f4ec9c42892 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:03.909263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221568bb-928e-4e05-99fc-8f4ec9c42892 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:03.910611 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221568bb-928e-4e05-99fc-8f4ec9c42892 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:03.920052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-221568bb-928e-4e05-99fc-8f4ec9c42892 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:43:03.920052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:03.920052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:43:03.920052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:43:03.920052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:43:03.920052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:43:03.920052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:43:03.920052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:03.920052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:43:03.920052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:03.920052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:03.920052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:03.920052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:03.920052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:03.920052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:03.920052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:03.921392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:03.921392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:03.921392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:03.921392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:03.921392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:03.921392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:03.921392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:03.921392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:43:03.921392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:43:03.921392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:43:03.921392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:43:03.921392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:03.921392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:03.921392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:03.921392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:03.921392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:03.921392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:03.922320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:03.922320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:03.922320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:03.922320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:03.922320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:03.922320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:03.922320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:03.922320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:03.922320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:03.922320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:03.922320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:43:03.922320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:03.922320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:43:03.922320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:03.922320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 420/852] 10.222.0.67 () {56 vars in 1132 bytes} [Wed Aug 24 08:43:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:03.923158 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221568bb-928e-4e05-99fc-8f4ec9c42892 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:03.962142 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e292d3-cd85-4eb8-b26a-6a2aa118f80e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:03.997074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:04.002560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc901f1b-5af3-45ab-80c3-9d400955853f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:04.003166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc901f1b-5af3-45ab-80c3-9d400955853f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:04.003515 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc901f1b-5af3-45ab-80c3-9d400955853f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:04.003788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc901f1b-5af3-45ab-80c3-9d400955853f None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:04.004508 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc901f1b-5af3-45ab-80c3-9d400955853f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:04.004811 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc901f1b-5af3-45ab-80c3-9d400955853f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:04.006284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc901f1b-5af3-45ab-80c3-9d400955853f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:04.010692 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:04.021805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 433/853] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:43:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:04.023594 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b6584-473b-4435-be25-ffdae463a8ed None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:04.023878 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b6584-473b-4435-be25-ffdae463a8ed None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:04.024195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b6584-473b-4435-be25-ffdae463a8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:04.043285 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221568bb-928e-4e05-99fc-8f4ec9c42892 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:04.043857 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e44b6584-473b-4435-be25-ffdae463a8ed None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:04.045085 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e44b6584-473b-4435-be25-ffdae463a8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:04.085528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e81be3-f51b-490e-b97b-ad3a3be6421a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:04.086263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e81be3-f51b-490e-b97b-ad3a3be6421a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:04.086543 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e81be3-f51b-490e-b97b-ad3a3be6421a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:04.086813 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e81be3-f51b-490e-b97b-ad3a3be6421a None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:04.087476 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e81be3-f51b-490e-b97b-ad3a3be6421a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:04.087780 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e81be3-f51b-490e-b97b-ad3a3be6421a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:04.089247 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e81be3-f51b-490e-b97b-ad3a3be6421a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:04.094600 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 421/854] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:43:03 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:04.095497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e81be3-f51b-490e-b97b-ad3a3be6421a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:04.141627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 434/855] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:04.142896 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b6584-473b-4435-be25-ffdae463a8ed None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:04.292636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e81be3-f51b-490e-b97b-ad3a3be6421a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:04.323340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b6584-473b-4435-be25-ffdae463a8ed None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:04.331747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:04.344341 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07f990-0447-49cf-8e6e-ab1b7d0a3ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:04.345376 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07f990-0447-49cf-8e6e-ab1b7d0a3ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:04.345740 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07f990-0447-49cf-8e6e-ab1b7d0a3ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:04.346076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07f990-0447-49cf-8e6e-ab1b7d0a3ad3 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:04.346800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07f990-0447-49cf-8e6e-ab1b7d0a3ad3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:04.347174 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07f990-0447-49cf-8e6e-ab1b7d0a3ad3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:04.348996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07f990-0447-49cf-8e6e-ab1b7d0a3ad3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:04.359148 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:04.361610 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 422/856] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 24 08:43:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:04.369186 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07f990-0447-49cf-8e6e-ab1b7d0a3ad3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:04.372974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752e5f6-2ef2-42a3-8959-5597124b1180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:04.373926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752e5f6-2ef2-42a3-8959-5597124b1180 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:04.374206 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752e5f6-2ef2-42a3-8959-5597124b1180 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:04.374550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752e5f6-2ef2-42a3-8959-5597124b1180 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:04.375299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752e5f6-2ef2-42a3-8959-5597124b1180 None admin] RBAC: Adding query filter params (role.id=b13ffa72344c4856901b1ba003e273fd, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=46b1417d4ac24c8982b2052f4e99042a) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:04.375611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752e5f6-2ef2-42a3-8959-5597124b1180 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:04.377097 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752e5f6-2ef2-42a3-8959-5597124b1180 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:04.384688 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 435/857] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:43:04 2022] GET /identity/v3/role_assignments?user.id=46b1417d4ac24c8982b2052f4e99042a&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=b13ffa72344c4856901b1ba003e273fd => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:04.452231 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752e5f6-2ef2-42a3-8959-5597124b1180 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:04.576766 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07f990-0447-49cf-8e6e-ab1b7d0a3ad3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:04.608055 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752e5f6-2ef2-42a3-8959-5597124b1180 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:04.648467 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbfe06-b87b-447c-922f-d2099fc925c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:04.649345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbfe06-b87b-447c-922f-d2099fc925c1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:04.649683 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbfe06-b87b-447c-922f-d2099fc925c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:04.650027 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbfe06-b87b-447c-922f-d2099fc925c1 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:04.651175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbfe06-b87b-447c-922f-d2099fc925c1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:04.651564 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbfe06-b87b-447c-922f-d2099fc925c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:04.653070 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbfe06-b87b-447c-922f-d2099fc925c1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:04.659197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 423/858] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 24 08:43:04 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:04.667930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028fc5d3-bf87-41fb-8c08-8205a2e11114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:04.668780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028fc5d3-bf87-41fb-8c08-8205a2e11114 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:04.669162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028fc5d3-bf87-41fb-8c08-8205a2e11114 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:04.669538 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028fc5d3-bf87-41fb-8c08-8205a2e11114 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:04.682808 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028fc5d3-bf87-41fb-8c08-8205a2e11114 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:43:04.688642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-028fc5d3-bf87-41fb-8c08-8205a2e11114 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:43:04.688642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:04.688642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:04.688642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:04.688642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:04.688642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:04.688642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:04.688642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:04.688642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:04.688642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:04.688642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:04.688642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:04.688642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:43:04.688642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:43:04.688642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:43:04.688642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:43:04.689778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:43:04.689778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:43:04.689778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:43:04.689778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:43:04.689778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:04.689778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:04.689778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:04.689778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:04.689778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:04.689778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:04.689778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:04.689778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:04.689778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:04.689778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:04.689778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:04.689778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:04.690628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:04.690628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:04.690628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:04.690628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:04.690628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:43:04.690628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:43:04.690628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:43:04.690628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:43:04.690628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:43:04.690628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:43:04.690628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:43:04.690628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:04.691298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 436/859] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:43:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:43:04.702376 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbfe06-b87b-447c-922f-d2099fc925c1 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:04.829794 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:04.901398 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbfe06-b87b-447c-922f-d2099fc925c1 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:04.966233 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3302b-f627-4c59-bcf8-b012da9cc98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:04.967262 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3302b-f627-4c59-bcf8-b012da9cc98a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:04.967727 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3302b-f627-4c59-bcf8-b012da9cc98a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:04.968233 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3302b-f627-4c59-bcf8-b012da9cc98a None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:04.969381 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3302b-f627-4c59-bcf8-b012da9cc98a None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:04.969920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3302b-f627-4c59-bcf8-b012da9cc98a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:04.972675 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3302b-f627-4c59-bcf8-b012da9cc98a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:04.993662 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 424/860] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:43:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:04.999013 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028fc5d3-bf87-41fb-8c08-8205a2e11114 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:05.095626 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:05.158411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028fc5d3-bf87-41fb-8c08-8205a2e11114 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:05.198078 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924c5df-6305-481b-9ae9-937f3f3fc6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:05.199100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924c5df-6305-481b-9ae9-937f3f3fc6ac None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:05.199589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924c5df-6305-481b-9ae9-937f3f3fc6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:05.200088 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924c5df-6305-481b-9ae9-937f3f3fc6ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:05.213715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a924c5df-6305-481b-9ae9-937f3f3fc6ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:05.215523 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a924c5df-6305-481b-9ae9-937f3f3fc6ac None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:05.231271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-a924c5df-6305-481b-9ae9-937f3f3fc6ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:05.231271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:05.231271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:05.231271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:05.231271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:05.231271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:05.231271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:05.231271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:05.231271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:05.231271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:05.231271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:05.231271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:05.231271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:43:05.231271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:43:05.231271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:43:05.231271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:43:05.232318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:05.232318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:05.232318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:05.232318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:05.232318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:05.232318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:05.232318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:05.232318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:05.232318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:05.232318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:05.232318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:05.232318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:05.232318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:05.232318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:05.232318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:05.232318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:05.232318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:43:05.233223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:43:05.233223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:05.233223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:05.235343 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 437/861] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:05.245214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3302b-f627-4c59-bcf8-b012da9cc98a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:05.423024 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3302b-f627-4c59-bcf8-b012da9cc98a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:05.455101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:05.490121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1c8df-c70a-45c8-9257-9343f735eefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:05.491892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1c8df-c70a-45c8-9257-9343f735eefc None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:05.492428 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1c8df-c70a-45c8-9257-9343f735eefc None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:05.492915 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1c8df-c70a-45c8-9257-9343f735eefc None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:05.494196 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1c8df-c70a-45c8-9257-9343f735eefc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:05.494745 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1c8df-c70a-45c8-9257-9343f735eefc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:05.497433 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1c8df-c70a-45c8-9257-9343f735eefc None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:05.520816 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 425/862] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:05.531140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924c5df-6305-481b-9ae9-937f3f3fc6ac None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:05.722226 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924c5df-6305-481b-9ae9-937f3f3fc6ac None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:05.726096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:05.789309 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8387b45-84ad-4f6b-ab31-296b6ae164dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:05.790392 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8387b45-84ad-4f6b-ab31-296b6ae164dd None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:05.790884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8387b45-84ad-4f6b-ab31-296b6ae164dd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:05.791371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8387b45-84ad-4f6b-ab31-296b6ae164dd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:05.807272 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8387b45-84ad-4f6b-ab31-296b6ae164dd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:05.810026 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8387b45-84ad-4f6b-ab31-296b6ae164dd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:05.833266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d8387b45-84ad-4f6b-ab31-296b6ae164dd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:43:05.833266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:05.833266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:05.833266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:05.833266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:05.833266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:05.833266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:05.833266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:05.833266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:05.833266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:05.833266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:05.833266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:05.833266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:43:05.833266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:43:05.833266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:43:05.833266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:43:05.834360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:05.834360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:05.834360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:43:05.834360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:05.834360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:43:05.834360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:05.834360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:05.834360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:05.834360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:05.834360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:05.834360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:05.834360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:05.834360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:05.834360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:05.834360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:05.834360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:05.834360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:05.834360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:05.835397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:05.835397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:05.835397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:43:05.835397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:43:05.835397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:43:05.835397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:43:05.835397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:43:05.835397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:43:05.835397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:43:05.835397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:05.836495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 438/863] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 24 08:43:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:43:05.842952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1c8df-c70a-45c8-9257-9343f735eefc None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:06.002000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1c8df-c70a-45c8-9257-9343f735eefc None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:06.042664 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9b01c-619f-47c4-af16-0270ea78d834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:06.043326 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9b01c-619f-47c4-af16-0270ea78d834 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:06.043618 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9b01c-619f-47c4-af16-0270ea78d834 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:06.043947 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9b01c-619f-47c4-af16-0270ea78d834 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:06.044905 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec9b01c-619f-47c4-af16-0270ea78d834 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:06.045210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec9b01c-619f-47c4-af16-0270ea78d834 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:06.046726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec9b01c-619f-47c4-af16-0270ea78d834 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:06.066723 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 426/864] 10.222.0.67 () {56 vars in 1149 bytes} [Wed Aug 24 08:43:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:06.072686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8387b45-84ad-4f6b-ab31-296b6ae164dd None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:06.199753 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8387b45-84ad-4f6b-ab31-296b6ae164dd None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:06.236602 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9f8de-7d9c-40aa-b1ea-f052642c7012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:06.237216 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9f8de-7d9c-40aa-b1ea-f052642c7012 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:06.237490 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9f8de-7d9c-40aa-b1ea-f052642c7012 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:06.237755 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9f8de-7d9c-40aa-b1ea-f052642c7012 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:06.271753 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:06.323055 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9f8de-7d9c-40aa-b1ea-f052642c7012 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:06.325885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9f8de-7d9c-40aa-b1ea-f052642c7012 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:06.467972 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 439/865] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:43:06 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:43:06.473754 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9b01c-619f-47c4-af16-0270ea78d834 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:06.536047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:06.593519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d45b97-5ecc-4bce-bc1a-dee0bb80181b None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:06.594515 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d45b97-5ecc-4bce-bc1a-dee0bb80181b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:06.594968 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d45b97-5ecc-4bce-bc1a-dee0bb80181b None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:06.599212 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 440/866] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:06.606715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84a22b-4ed8-42d3-9a46-7b7a52aab190 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:06.607204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84a22b-4ed8-42d3-9a46-7b7a52aab190 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:06.607687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84a22b-4ed8-42d3-9a46-7b7a52aab190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:06.639331 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8b84a22b-4ed8-42d3-9a46-7b7a52aab190 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:06.639663 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9b01c-619f-47c4-af16-0270ea78d834 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:06.641434 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8b84a22b-4ed8-42d3-9a46-7b7a52aab190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:06.677355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cfb30-3ae1-49af-8518-3f856a77458c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:06.677977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cfb30-3ae1-49af-8518-3f856a77458c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:06.678253 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cfb30-3ae1-49af-8518-3f856a77458c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:06.679324 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cfb30-3ae1-49af-8518-3f856a77458c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:06.690263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511cfb30-3ae1-49af-8518-3f856a77458c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:06.691719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511cfb30-3ae1-49af-8518-3f856a77458c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:06.709062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-511cfb30-3ae1-49af-8518-3f856a77458c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:06.709062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:06.709062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:06.709062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:06.709062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:06.709062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:06.709062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:06.709062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:06.709062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:06.709062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:06.709062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:06.709062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:06.709062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:43:06.709062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:43:06.709062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:43:06.709062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:43:06.710651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:06.710651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:06.710651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:06.710651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:06.710651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:06.710651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:06.710651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:06.710651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:06.710651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:06.710651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:06.710651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:06.710651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:06.710651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:06.710651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:06.710651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:06.710651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:06.710651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:43:06.711692 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:43:06.711692 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:06.711692 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:06.711692 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 427/867] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:06.712399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f98ec-5c1a-4405-bea8-c533c87f4b30 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:06.712641 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f98ec-5c1a-4405-bea8-c533c87f4b30 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:06.720142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f98ec-5c1a-4405-bea8-c533c87f4b30 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:06.720142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 428/868] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:06.720142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f98ec-5c1a-4405-bea8-c533c87f4b30 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:06.755228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:06.768977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 441/869] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:43:06.775184 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8c6d7-ab4a-40e3-a7f3-21609f8e25c7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:06.775990 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8c6d7-ab4a-40e3-a7f3-21609f8e25c7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:06.776445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8c6d7-ab4a-40e3-a7f3-21609f8e25c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:06.801294 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-62e8c6d7-ab4a-40e3-a7f3-21609f8e25c7 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:06.802686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-62e8c6d7-ab4a-40e3-a7f3-21609f8e25c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:06.845010 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:06.877754 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f98ec-5c1a-4405-bea8-c533c87f4b30 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:06.922066 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 442/870] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:06.924725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92ae6a-5fae-4f32-bf4a-55077bea6cc6 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:06.925463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92ae6a-5fae-4f32-bf4a-55077bea6cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:06.926220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92ae6a-5fae-4f32-bf4a-55077bea6cc6 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:06.936780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 443/871] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:06 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:06.939369 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e7877-9f0b-464f-816a-d7435148d465 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:06.940171 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e7877-9f0b-464f-816a-d7435148d465 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:06.940935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e7877-9f0b-464f-816a-d7435148d465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:06.950752 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37ea5f-7f2b-4ecd-a856-32347f7b5954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:06.951799 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37ea5f-7f2b-4ecd-a856-32347f7b5954 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:06.952333 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37ea5f-7f2b-4ecd-a856-32347f7b5954 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:06.952884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37ea5f-7f2b-4ecd-a856-32347f7b5954 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:06.954372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37ea5f-7f2b-4ecd-a856-32347f7b5954 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:06.954908 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37ea5f-7f2b-4ecd-a856-32347f7b5954 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:06.957701 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37ea5f-7f2b-4ecd-a856-32347f7b5954 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:06.973059 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-633e7877-9f0b-464f-816a-d7435148d465 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:06.976248 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-633e7877-9f0b-464f-816a-d7435148d465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:06.988674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 429/872] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:06.992591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37ea5f-7f2b-4ecd-a856-32347f7b5954 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:07.095079 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 444/873] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:07.097116 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e7877-9f0b-464f-816a-d7435148d465 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:07.187302 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:07.187501 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37ea5f-7f2b-4ecd-a856-32347f7b5954 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:07.247028 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3dbaf-5306-4517-9335-f16061f2613b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:07.247683 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3dbaf-5306-4517-9335-f16061f2613b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:07.247964 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3dbaf-5306-4517-9335-f16061f2613b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:07.248271 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3dbaf-5306-4517-9335-f16061f2613b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:07.256308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3dbaf-5306-4517-9335-f16061f2613b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:07.258020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3dbaf-5306-4517-9335-f16061f2613b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:07.264378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e7877-9f0b-464f-816a-d7435148d465 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:07.270549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-44e3dbaf-5306-4517-9335-f16061f2613b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:07.270549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:07.270549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:07.270549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:07.270549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:07.270549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:07.270549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:07.270549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:07.270549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:07.270549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:07.270549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:07.270549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:07.270549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:43:07.270549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:43:07.270549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:43:07.271530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:43:07.271530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:07.271530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:07.271530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:07.271530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:07.271530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:07.271530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:07.271530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:07.271530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:07.271530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:07.271530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:07.271530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:07.271530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:07.271530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:07.271530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:07.271530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:07.271530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:07.272527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:43:07.272527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:43:07.272527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:43:07.272527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:43:07.272527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:43:07.272527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:43:07.272527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:07.272527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:07.272527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 430/874] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:43:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:07.273165 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3dbaf-5306-4517-9335-f16061f2613b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:07.283973 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:07.304524 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453d2aa-73b7-495c-b691-ccd892475031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:07.305522 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453d2aa-73b7-495c-b691-ccd892475031 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:07.305792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453d2aa-73b7-495c-b691-ccd892475031 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:07.306217 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453d2aa-73b7-495c-b691-ccd892475031 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:07.306879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453d2aa-73b7-495c-b691-ccd892475031 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:07.308280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453d2aa-73b7-495c-b691-ccd892475031 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:07.332182 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 445/875] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 24 08:43:07 2022] POST /identity/v3/services => generated 251 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:43:07.332182 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453d2aa-73b7-495c-b691-ccd892475031 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:07.405176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3dbaf-5306-4517-9335-f16061f2613b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:07.452569 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab443577-9686-46ef-8974-9a834371932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:07.453227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab443577-9686-46ef-8974-9a834371932e None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:07.453510 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab443577-9686-46ef-8974-9a834371932e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:07.453844 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab443577-9686-46ef-8974-9a834371932e None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:07.454446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab443577-9686-46ef-8974-9a834371932e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:07.456009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab443577-9686-46ef-8974-9a834371932e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:07.464818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453d2aa-73b7-495c-b691-ccd892475031 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:07.480208 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 431/876] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 24 08:43:07 2022] POST /identity/v3/services => generated 269 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:43:07.480208 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab443577-9686-46ef-8974-9a834371932e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:07.487301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:07.535757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92d224-1f4a-4782-b5a0-3a7b02c9923b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:07.536747 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92d224-1f4a-4782-b5a0-3a7b02c9923b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:07.537250 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92d224-1f4a-4782-b5a0-3a7b02c9923b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:07.537524 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92d224-1f4a-4782-b5a0-3a7b02c9923b None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:07.538323 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92d224-1f4a-4782-b5a0-3a7b02c9923b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:07.538790 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92d224-1f4a-4782-b5a0-3a7b02c9923b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:07.540995 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92d224-1f4a-4782-b5a0-3a7b02c9923b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:07.560164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 446/877] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:43:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:07.565669 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92d224-1f4a-4782-b5a0-3a7b02c9923b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:07.576703 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:07.642860 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab443577-9686-46ef-8974-9a834371932e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:07.699010 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea057a7-cb66-4a13-9794-dc95fc9fea84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:07.699588 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea057a7-cb66-4a13-9794-dc95fc9fea84 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:07.699868 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea057a7-cb66-4a13-9794-dc95fc9fea84 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:07.700177 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea057a7-cb66-4a13-9794-dc95fc9fea84 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:07.700917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea057a7-cb66-4a13-9794-dc95fc9fea84 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:07.702795 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea057a7-cb66-4a13-9794-dc95fc9fea84 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:07.715037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-5ea057a7-cb66-4a13-9794-dc95fc9fea84 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:43:07.715037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:07.715037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:43:07.715037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:43:07.715037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:43:07.715037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:43:07.715037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:43:07.715037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:07.715037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:43:07.715037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:07.715037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:07.715037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:07.715037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:07.715037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:07.715037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:07.715037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:07.715037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:07.716122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:07.716122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:07.716122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:07.716122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:07.716122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:07.716122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:07.716122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:43:07.716122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:43:07.716122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:43:07.716122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:43:07.716122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:07.716122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:07.716122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:07.716122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:07.716122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:07.716122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:07.716988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:07.716988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:07.716988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:07.716988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:07.716988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:07.716988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:07.716988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:07.716988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:07.716988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:07.716988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:07.716988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:43:07.716988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:07.716988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:43:07.716988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:07.716988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 432/878] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 24 08:43:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:07.723072 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea057a7-cb66-4a13-9794-dc95fc9fea84 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:07.728861 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92d224-1f4a-4782-b5a0-3a7b02c9923b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:07.771703 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc51d4-bd56-481a-aec3-7d0c79d5d3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:07.773133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc51d4-bd56-481a-aec3-7d0c79d5d3ee None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:07.773654 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc51d4-bd56-481a-aec3-7d0c79d5d3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:07.774022 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc51d4-bd56-481a-aec3-7d0c79d5d3ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:07.774816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc51d4-bd56-481a-aec3-7d0c79d5d3ee None admin] RBAC: Adding query filter params (role.id=9aa748c8ea2542e6987e0b93190ea4e6, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=b11abed8d4424a1e9957c3dc9aefadd5) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:07.775126 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc51d4-bd56-481a-aec3-7d0c79d5d3ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:07.776640 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc51d4-bd56-481a-aec3-7d0c79d5d3ee None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:07.783797 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 447/879] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:43:07 2022] GET /identity/v3/role_assignments?user.id=b11abed8d4424a1e9957c3dc9aefadd5&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=9aa748c8ea2542e6987e0b93190ea4e6 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:07.862757 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea057a7-cb66-4a13-9794-dc95fc9fea84 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:07.905256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:07.908679 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae5e6b5-b334-47f1-a652-3a0404a22741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:07.909325 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5e6b5-b334-47f1-a652-3a0404a22741 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:07.909600 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5e6b5-b334-47f1-a652-3a0404a22741 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:07.909874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5e6b5-b334-47f1-a652-3a0404a22741 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:07.910591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5e6b5-b334-47f1-a652-3a0404a22741 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:07.911304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5e6b5-b334-47f1-a652-3a0404a22741 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:07.913511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5e6b5-b334-47f1-a652-3a0404a22741 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:07.919829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 433/880] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 24 08:43:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:07.925761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc51d4-bd56-481a-aec3-7d0c79d5d3ee None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:07.968909 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:08.053727 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc51d4-bd56-481a-aec3-7d0c79d5d3ee None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:08.098214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e010ea-5fd9-4966-8d47-2e69d2033ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:08.098834 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e010ea-5fd9-4966-8d47-2e69d2033ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:08.099112 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e010ea-5fd9-4966-8d47-2e69d2033ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:08.099381 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e010ea-5fd9-4966-8d47-2e69d2033ce5 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:08.100105 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e010ea-5fd9-4966-8d47-2e69d2033ce5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:08.100559 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e010ea-5fd9-4966-8d47-2e69d2033ce5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:08.102158 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e010ea-5fd9-4966-8d47-2e69d2033ce5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:08.108058 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 448/881] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 24 08:43:07 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:09.467347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22844c5-1813-453d-9e8d-d6368a8e94be None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:09.468253 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22844c5-1813-453d-9e8d-d6368a8e94be None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:09.468675 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22844c5-1813-453d-9e8d-d6368a8e94be None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:09.470463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 434/882] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:09.477098 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3cd51-c60b-4653-ae68-5e460c375abf None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:09.477824 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3cd51-c60b-4653-ae68-5e460c375abf None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:09.477824 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3cd51-c60b-4653-ae68-5e460c375abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:09.497427 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7be3cd51-c60b-4653-ae68-5e460c375abf None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:09.498635 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7be3cd51-c60b-4653-ae68-5e460c375abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:09.576062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 449/883] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:09 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:09.789082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12430284-84e9-4d32-88ec-954d259e069e None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:09.789557 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12430284-84e9-4d32-88ec-954d259e069e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:09.790288 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12430284-84e9-4d32-88ec-954d259e069e None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:09.793177 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 435/884] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:09.801918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4e240-4e10-4bdd-bff9-3e33dc88d639 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:09.802694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4e240-4e10-4bdd-bff9-3e33dc88d639 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:09.803375 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4e240-4e10-4bdd-bff9-3e33dc88d639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:09.833897 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f9e4e240-4e10-4bdd-bff9-3e33dc88d639 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:09.836876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f9e4e240-4e10-4bdd-bff9-3e33dc88d639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:09.861180 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12430284-84e9-4d32-88ec-954d259e069e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:09.874739 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:09.965296 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 450/885] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:09 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:10.031308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12430284-84e9-4d32-88ec-954d259e069e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:10.073012 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe1961-57ca-4752-be3d-90962434b599 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:10.073576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe1961-57ca-4752-be3d-90962434b599 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:10.074002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe1961-57ca-4752-be3d-90962434b599 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:10.075841 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 451/886] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:43:10.081813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34391e7-1725-42f4-8c7c-2d1d429c2784 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:10.082355 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34391e7-1725-42f4-8c7c-2d1d429c2784 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:10.082794 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34391e7-1725-42f4-8c7c-2d1d429c2784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:10.091939 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2598228-fca5-4c41-946e-ce5daddaf7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:10.093300 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2598228-fca5-4c41-946e-ce5daddaf7aa None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:10.093854 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2598228-fca5-4c41-946e-ce5daddaf7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:10.094343 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2598228-fca5-4c41-946e-ce5daddaf7aa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:10.095426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2598228-fca5-4c41-946e-ce5daddaf7aa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:10.098242 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2598228-fca5-4c41-946e-ce5daddaf7aa None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:10.104291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b34391e7-1725-42f4-8c7c-2d1d429c2784 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:10.105756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b34391e7-1725-42f4-8c7c-2d1d429c2784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:10.115229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-c2598228-fca5-4c41-946e-ce5daddaf7aa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:43:10.115229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:10.115229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:43:10.115229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:43:10.115229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:43:10.115229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:43:10.115229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:43:10.115229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:10.115229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:43:10.115229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:10.115229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:10.115229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:10.115229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:10.115229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:10.115229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:10.115229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:10.115229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:10.116668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:10.116668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:10.116668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:10.116668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:10.116668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:10.116668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:10.116668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:43:10.116668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:43:10.116668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:43:10.116668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:43:10.116668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:10.116668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:10.116668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:10.116668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:10.116668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:10.116668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:10.117671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:10.117671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:10.117671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:10.117671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:10.117671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:10.117671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:10.117671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:10.117671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:10.117671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:10.117671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:10.117671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:43:10.117671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:10.117671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:43:10.117671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:10.117671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 436/887] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:43:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:10.123166 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db0589-37ae-463f-8751-ee13cc6d5d37 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:10.123458 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db0589-37ae-463f-8751-ee13cc6d5d37 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:10.123744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db0589-37ae-463f-8751-ee13cc6d5d37 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:10.125269 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 437/888] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:10.125847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db0589-37ae-463f-8751-ee13cc6d5d37 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:10.200428 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:10.216163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 452/889] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:10 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:43:10.216163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a76400-55da-42cb-9260-6b8995fbd4c1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:10.216163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a76400-55da-42cb-9260-6b8995fbd4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:10.216163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a76400-55da-42cb-9260-6b8995fbd4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:10.259991 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a6a76400-55da-42cb-9260-6b8995fbd4c1 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:10.261240 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a6a76400-55da-42cb-9260-6b8995fbd4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:10.268904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db0589-37ae-463f-8751-ee13cc6d5d37 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:10.306750 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428f9e2-5353-49bd-b8ae-cc66d12bbe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:10.307259 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428f9e2-5353-49bd-b8ae-cc66d12bbe4e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:10.307596 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428f9e2-5353-49bd-b8ae-cc66d12bbe4e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:10.307860 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428f9e2-5353-49bd-b8ae-cc66d12bbe4e None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:10.308452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428f9e2-5353-49bd-b8ae-cc66d12bbe4e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:10.308746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428f9e2-5353-49bd-b8ae-cc66d12bbe4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:10.310104 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428f9e2-5353-49bd-b8ae-cc66d12bbe4e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:10.315198 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 438/890] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:43:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:10.316078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428f9e2-5353-49bd-b8ae-cc66d12bbe4e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:10.360672 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 453/891] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:10 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:10.361899 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a76400-55da-42cb-9260-6b8995fbd4c1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:10.390543 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:10.491786 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428f9e2-5353-49bd-b8ae-cc66d12bbe4e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:10.492420 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a76400-55da-42cb-9260-6b8995fbd4c1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:10.520740 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:10.529462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf378a-2056-4741-9050-a8ff5d8126bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:10.530112 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf378a-2056-4741-9050-a8ff5d8126bf None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:10.530500 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf378a-2056-4741-9050-a8ff5d8126bf None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:10.530655 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf378a-2056-4741-9050-a8ff5d8126bf None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:10.531295 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf378a-2056-4741-9050-a8ff5d8126bf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:10.531594 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf378a-2056-4741-9050-a8ff5d8126bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:10.533039 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf378a-2056-4741-9050-a8ff5d8126bf None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:10.547978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 454/892] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:43:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:10.547978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf378a-2056-4741-9050-a8ff5d8126bf None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:10.551892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7950827-9f87-4501-8e21-3b20d027902e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:10.557372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7950827-9f87-4501-8e21-3b20d027902e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:10.558542 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7950827-9f87-4501-8e21-3b20d027902e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:10.559034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7950827-9f87-4501-8e21-3b20d027902e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:10.560722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7950827-9f87-4501-8e21-3b20d027902e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:10.563695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7950827-9f87-4501-8e21-3b20d027902e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:10.577800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-a7950827-9f87-4501-8e21-3b20d027902e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:43:10.577800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:10.577800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:43:10.577800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:43:10.577800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:43:10.577800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:43:10.577800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:43:10.577800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:10.577800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:43:10.577800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:10.577800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:10.577800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:10.577800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:10.577800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:10.577800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:10.577800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:10.580658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:10.580658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:10.580658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:10.580658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:10.580658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:10.580658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:10.580658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:10.580658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:43:10.580658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:43:10.580658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:43:10.580658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:43:10.580658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:10.580658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:10.580658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:10.580658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:10.580658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:10.580658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:10.582289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:10.582289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:10.582289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:10.582289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:10.582289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:10.582289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:10.582289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:10.582289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:10.582289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:10.582289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:10.582289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:43:10.582289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:10.582289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:43:10.582289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:10.582289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 439/893] 10.222.0.67 () {56 vars in 1132 bytes} [Wed Aug 24 08:43:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:10.583721 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7950827-9f87-4501-8e21-3b20d027902e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:10.684265 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:10.709067 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf378a-2056-4741-9050-a8ff5d8126bf None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:10.717966 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7950827-9f87-4501-8e21-3b20d027902e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:10.753583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b9989-1aa5-4c0a-bfa3-e26089cd968d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:10.754233 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b9989-1aa5-4c0a-bfa3-e26089cd968d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:10.754621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b9989-1aa5-4c0a-bfa3-e26089cd968d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:10.754757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b9989-1aa5-4c0a-bfa3-e26089cd968d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:10.766446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e2a67-feed-4216-83a0-e641498c07dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:10.767069 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e2a67-feed-4216-83a0-e641498c07dd None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:10.767236 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b9989-1aa5-4c0a-bfa3-e26089cd968d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:43:10.767374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e2a67-feed-4216-83a0-e641498c07dd None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:10.767602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e2a67-feed-4216-83a0-e641498c07dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:10.768615 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e2a67-feed-4216-83a0-e641498c07dd None admin] RBAC: Adding query filter params (interface=public, service_id=a65f27f5283b4787aa7e89a0870d75df, region_id=RegionOne) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:10.768785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e2a67-feed-4216-83a0-e641498c07dd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:10.770032 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e2a67-feed-4216-83a0-e641498c07dd None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:10.771871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-433b9989-1aa5-4c0a-bfa3-e26089cd968d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:43:10.771871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:10.771871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:10.771871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:10.771871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:10.771871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:10.771871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:10.771871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:10.771871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:10.771871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:10.771871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:10.771871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:10.771871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:43:10.771871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:43:10.771871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:43:10.771871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:43:10.773480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:43:10.773480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:43:10.773480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:43:10.773480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:43:10.773480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:10.773480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:10.773480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:10.773480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:10.773480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:10.773480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:10.773480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:10.773480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:10.773480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:10.773480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:10.773480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:10.773480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:10.774450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:10.774450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:10.774450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:10.774450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:10.774450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:43:10.774450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:43:10.774450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:43:10.774450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:43:10.774450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:43:10.774450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:43:10.774450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:43:10.774450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:10.774450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 455/894] 10.222.0.67 () {56 vars in 1120 bytes} [Wed Aug 24 08:43:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:10.775148 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b9989-1aa5-4c0a-bfa3-e26089cd968d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:10.782926 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 440/895] 10.222.0.67 () {56 vars in 1251 bytes} [Wed Aug 24 08:43:10 2022] GET /identity/v3/endpoints?service_id=a65f27f5283b4787aa7e89a0870d75df&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:10.782926 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e2a67-feed-4216-83a0-e641498c07dd None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:10.838310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:10.890813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b9989-1aa5-4c0a-bfa3-e26089cd968d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:10.897996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e2a67-feed-4216-83a0-e641498c07dd None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:10.936688 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10cbc25-e6c2-4404-84da-622af29a25ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:10.937249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047b1e0-2243-4768-ad0b-cf95b880c95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:10.937556 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10cbc25-e6c2-4404-84da-622af29a25ea None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:10.937861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047b1e0-2243-4768-ad0b-cf95b880c95f None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:10.938026 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10cbc25-e6c2-4404-84da-622af29a25ea None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:10.938197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047b1e0-2243-4768-ad0b-cf95b880c95f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:10.938390 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047b1e0-2243-4768-ad0b-cf95b880c95f None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:10.938547 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10cbc25-e6c2-4404-84da-622af29a25ea None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:10.939050 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5047b1e0-2243-4768-ad0b-cf95b880c95f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:10.939279 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10cbc25-e6c2-4404-84da-622af29a25ea None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:10.939678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10cbc25-e6c2-4404-84da-622af29a25ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:10.940457 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5047b1e0-2243-4768-ad0b-cf95b880c95f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:10.941266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10cbc25-e6c2-4404-84da-622af29a25ea None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:10.947693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 456/896] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 24 08:43:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:10.948784 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10cbc25-e6c2-4404-84da-622af29a25ea None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:10.954063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:10.961983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 441/897] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 24 08:43:10 2022] POST /identity/v3/services => generated 241 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:43:10.962863 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047b1e0-2243-4768-ad0b-cf95b880c95f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:11.102166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10cbc25-e6c2-4404-84da-622af29a25ea None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:11.102358 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047b1e0-2243-4768-ad0b-cf95b880c95f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:11.104149 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:11.144943 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc7c64-5882-4c9c-9428-abb50d662e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:11.145654 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc7c64-5882-4c9c-9428-abb50d662e93 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:11.145984 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc7c64-5882-4c9c-9428-abb50d662e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:11.146236 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc7c64-5882-4c9c-9428-abb50d662e93 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:11.147066 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc7c64-5882-4c9c-9428-abb50d662e93 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:11.147940 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc7c64-5882-4c9c-9428-abb50d662e93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:11.149379 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc7c64-5882-4c9c-9428-abb50d662e93 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:11.150594 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0c7e3-a1d2-4f02-b30d-3a5aa28e2c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:11.151227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0c7e3-a1d2-4f02-b30d-3a5aa28e2c32 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:11.151508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0c7e3-a1d2-4f02-b30d-3a5aa28e2c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:11.151781 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0c7e3-a1d2-4f02-b30d-3a5aa28e2c32 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:11.152449 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0c7e3-a1d2-4f02-b30d-3a5aa28e2c32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:11.153880 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0c7e3-a1d2-4f02-b30d-3a5aa28e2c32 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:11.165518 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 457/898] 10.222.0.67 () {56 vars in 1115 bytes} [Wed Aug 24 08:43:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:11.165518 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc7c64-5882-4c9c-9428-abb50d662e93 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:11.172494 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 442/899] 10.222.0.67 () {56 vars in 1087 bytes} [Wed Aug 24 08:43:10 2022] GET /identity/v3/services => generated 1497 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:43:11.180251 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0c7e3-a1d2-4f02-b30d-3a5aa28e2c32 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:11.306911 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:11.324123 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0c7e3-a1d2-4f02-b30d-3a5aa28e2c32 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:11.361271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc7c64-5882-4c9c-9428-abb50d662e93 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:11.370888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57a862-892a-4fec-aaeb-45a8b5546687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:11.370888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57a862-892a-4fec-aaeb-45a8b5546687 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:11.371277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57a862-892a-4fec-aaeb-45a8b5546687 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:11.371567 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57a862-892a-4fec-aaeb-45a8b5546687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:11.390096 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57a862-892a-4fec-aaeb-45a8b5546687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:11.392434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57a862-892a-4fec-aaeb-45a8b5546687 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:11.414379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-4f57a862-892a-4fec-aaeb-45a8b5546687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:11.414379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:11.414379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:11.414379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:11.414379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:11.414379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:11.414379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:11.414379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:11.414379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:11.414379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:11.414379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:11.414379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:11.414379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:43:11.414379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:43:11.414379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:43:11.414379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:43:11.415705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:11.415705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:11.415705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:11.415705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:11.415705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:11.415705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:11.415705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:11.415705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:11.415705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:11.415705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:11.415705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:11.415705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:11.415705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:11.415705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:11.415705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:11.415705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:11.415705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:43:11.417200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:43:11.417200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:11.417200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:11.419935 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 443/900] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:11.431460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57a862-892a-4fec-aaeb-45a8b5546687 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:11.439290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e91733-791b-4600-81fb-2a9a894abd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:11.440671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e91733-791b-4600-81fb-2a9a894abd1c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:11.441227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e91733-791b-4600-81fb-2a9a894abd1c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:11.441694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e91733-791b-4600-81fb-2a9a894abd1c None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:11.442539 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e91733-791b-4600-81fb-2a9a894abd1c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:11.443060 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e91733-791b-4600-81fb-2a9a894abd1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:11.445063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e91733-791b-4600-81fb-2a9a894abd1c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:11.454100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 458/901] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 24 08:43:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:11.462061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e91733-791b-4600-81fb-2a9a894abd1c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:11.473457 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:11.602827 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e91733-791b-4600-81fb-2a9a894abd1c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:11.630877 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57a862-892a-4fec-aaeb-45a8b5546687 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:11.644189 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:11.651706 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545574a0-7b5b-460a-9cb3-7d092b66de93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:11.652364 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545574a0-7b5b-460a-9cb3-7d092b66de93 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:11.652677 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545574a0-7b5b-460a-9cb3-7d092b66de93 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:11.652942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545574a0-7b5b-460a-9cb3-7d092b66de93 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:11.653629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545574a0-7b5b-460a-9cb3-7d092b66de93 None admin] RBAC: Adding query filter params (interface=public, service_id=3c58a58e57c6433aa05f95e7f6e87593, region_id=RegionOne) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:11.653940 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545574a0-7b5b-460a-9cb3-7d092b66de93 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:11.655970 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545574a0-7b5b-460a-9cb3-7d092b66de93 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:11.663778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 459/902] 10.222.0.67 () {56 vars in 1251 bytes} [Wed Aug 24 08:43:11 2022] GET /identity/v3/endpoints?service_id=3c58a58e57c6433aa05f95e7f6e87593&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:11.673398 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545574a0-7b5b-460a-9cb3-7d092b66de93 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:11.706248 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953c7ef-4c60-49a6-bc9d-8f37e323904c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:11.707745 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953c7ef-4c60-49a6-bc9d-8f37e323904c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:11.708470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953c7ef-4c60-49a6-bc9d-8f37e323904c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:11.709215 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953c7ef-4c60-49a6-bc9d-8f37e323904c None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:11.710642 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953c7ef-4c60-49a6-bc9d-8f37e323904c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:11.711403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953c7ef-4c60-49a6-bc9d-8f37e323904c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:11.714802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953c7ef-4c60-49a6-bc9d-8f37e323904c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:11.736255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 444/903] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:11.754144 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953c7ef-4c60-49a6-bc9d-8f37e323904c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:11.833228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545574a0-7b5b-460a-9cb3-7d092b66de93 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:11.884074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:11.920144 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e942aa-f93a-43b0-8737-c4d285724eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:11.920144 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e942aa-f93a-43b0-8737-c4d285724eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:11.920144 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e942aa-f93a-43b0-8737-c4d285724eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:11.920144 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e942aa-f93a-43b0-8737-c4d285724eb5 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:11.920820 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e942aa-f93a-43b0-8737-c4d285724eb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:11.920942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953c7ef-4c60-49a6-bc9d-8f37e323904c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:11.921058 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e942aa-f93a-43b0-8737-c4d285724eb5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:11.936048 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 460/904] 10.222.0.67 () {56 vars in 1087 bytes} [Wed Aug 24 08:43:11 2022] GET /identity/v3/services => generated 1497 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:43:11.973211 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17133cb2-d471-414e-9e1f-bf4d7e7e9861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:11.973984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17133cb2-d471-414e-9e1f-bf4d7e7e9861 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:11.974265 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17133cb2-d471-414e-9e1f-bf4d7e7e9861 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:11.974538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17133cb2-d471-414e-9e1f-bf4d7e7e9861 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:11.984625 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17133cb2-d471-414e-9e1f-bf4d7e7e9861 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:11.986296 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17133cb2-d471-414e-9e1f-bf4d7e7e9861 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:12.002110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-17133cb2-d471-414e-9e1f-bf4d7e7e9861 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:43:12.002110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:12.002110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:12.002110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:12.002110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:12.002110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:12.002110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:12.002110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:12.002110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:12.002110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:12.002110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:12.002110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:12.002110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:43:12.002110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:43:12.002110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:43:12.002110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:43:12.003983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:12.003983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:12.003983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:43:12.003983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:12.003983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:43:12.003983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:12.003983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:12.003983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:12.003983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:12.003983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:12.003983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:12.003983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:12.003983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:12.003983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:12.003983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:12.003983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:12.003983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:12.003983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:12.005824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:12.005824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:12.005824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:43:12.005824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:43:12.005824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:43:12.005824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:43:12.005824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:43:12.005824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:43:12.005824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:43:12.005824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:12.005824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 445/905] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 24 08:43:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:12.017588 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e942aa-f93a-43b0-8737-c4d285724eb5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:12.105355 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:12.161248 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e942aa-f93a-43b0-8737-c4d285724eb5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:12.201097 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8f275-d209-49a9-99b8-564a714303fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:12.201731 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8f275-d209-49a9-99b8-564a714303fd None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:12.202048 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8f275-d209-49a9-99b8-564a714303fd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:12.202321 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8f275-d209-49a9-99b8-564a714303fd None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:12.203077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb8f275-d209-49a9-99b8-564a714303fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:12.203496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb8f275-d209-49a9-99b8-564a714303fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:12.204996 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb8f275-d209-49a9-99b8-564a714303fd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:12.226492 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 461/906] 10.222.0.67 () {56 vars in 1149 bytes} [Wed Aug 24 08:43:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:12.232839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17133cb2-d471-414e-9e1f-bf4d7e7e9861 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:12.365317 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17133cb2-d471-414e-9e1f-bf4d7e7e9861 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:12.411800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f3600-01c1-498d-a717-eb7e44eb2e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:12.412563 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f3600-01c1-498d-a717-eb7e44eb2e46 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:12.412926 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f3600-01c1-498d-a717-eb7e44eb2e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:12.413294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f3600-01c1-498d-a717-eb7e44eb2e46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:12.424250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:12.492022 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f3600-01c1-498d-a717-eb7e44eb2e46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:12.493708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f3600-01c1-498d-a717-eb7e44eb2e46 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:12.609380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 446/907] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:43:12 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:43:12.618441 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8f275-d209-49a9-99b8-564a714303fd None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:12.757941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8f275-d209-49a9-99b8-564a714303fd None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:12.776364 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:12.795830 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c53e8e-2071-4f10-9bc0-22aaccdcb1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:12.796503 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c53e8e-2071-4f10-9bc0-22aaccdcb1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:12.796781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c53e8e-2071-4f10-9bc0-22aaccdcb1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:12.797050 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c53e8e-2071-4f10-9bc0-22aaccdcb1f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:12.808338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c53e8e-2071-4f10-9bc0-22aaccdcb1f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:12.809927 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c53e8e-2071-4f10-9bc0-22aaccdcb1f5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:12.826161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-03c53e8e-2071-4f10-9bc0-22aaccdcb1f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:12.826161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:12.826161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:12.826161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:12.826161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:12.826161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:12.826161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:12.826161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:12.826161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:12.826161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:12.826161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:12.826161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:12.826161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:43:12.826161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:43:12.826161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:43:12.826161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:43:12.827871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:12.827871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:12.827871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:12.827871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:12.827871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:12.827871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:12.827871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:12.827871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:12.827871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:12.827871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:12.827871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:12.827871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:12.827871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:12.827871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:12.827871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:12.827871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:12.827871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:43:12.829018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:43:12.829018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:12.829018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:12.829018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 462/908] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:12.837261 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f3600-01c1-498d-a717-eb7e44eb2e46 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:12.885837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:12.956965 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f3600-01c1-498d-a717-eb7e44eb2e46 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:12.994796 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd6058-42f1-44db-86db-e5118a131067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:12.995338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd6058-42f1-44db-86db-e5118a131067 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:12.995618 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd6058-42f1-44db-86db-e5118a131067 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:12.995888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd6058-42f1-44db-86db-e5118a131067 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:12.996567 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd6058-42f1-44db-86db-e5118a131067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:12.996867 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd6058-42f1-44db-86db-e5118a131067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:12.998244 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd6058-42f1-44db-86db-e5118a131067 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:13.013069 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 447/909] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:13.021230 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c53e8e-2071-4f10-9bc0-22aaccdcb1f5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:13.170193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c53e8e-2071-4f10-9bc0-22aaccdcb1f5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:13.235006 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c55867-4b23-4f77-8dd6-413d4a7a0b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:13.236118 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c55867-4b23-4f77-8dd6-413d4a7a0b7a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:13.236637 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c55867-4b23-4f77-8dd6-413d4a7a0b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:13.237126 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c55867-4b23-4f77-8dd6-413d4a7a0b7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:13.250503 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c55867-4b23-4f77-8dd6-413d4a7a0b7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:13.253222 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c55867-4b23-4f77-8dd6-413d4a7a0b7a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:13.275586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-04c55867-4b23-4f77-8dd6-413d4a7a0b7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:13.275586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:13.275586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:13.275586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:13.275586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:13.275586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:13.275586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:13.275586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:13.275586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:13.275586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:13.275586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:13.275586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:13.275586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:43:13.275586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:43:13.275586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:43:13.276576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:43:13.276576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:13.276576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:13.276576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:13.276576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:13.276576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:13.276576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:13.276576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:13.276576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:13.276576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:13.276576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:13.276576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:13.276576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:13.276576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:13.276576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:13.276576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:13.276576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:13.277473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:43:13.277473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:43:13.277473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:43:13.277473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:43:13.277473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:43:13.277473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:43:13.277473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:13.277473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:13.278462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 463/910] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:43:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:13.289636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd6058-42f1-44db-86db-e5118a131067 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:13.418429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1c204-3dba-4fbe-a44e-9619c4463e07 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:13.418723 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1c204-3dba-4fbe-a44e-9619c4463e07 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:13.418956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1c204-3dba-4fbe-a44e-9619c4463e07 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:13.420653 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 464/911] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:13.426287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5c0a6-426d-4fc5-98db-5155e7fd3081 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:13.426600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5c0a6-426d-4fc5-98db-5155e7fd3081 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:13.427156 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5c0a6-426d-4fc5-98db-5155e7fd3081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:13.435265 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:13.439376 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd6058-42f1-44db-86db-e5118a131067 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:13.461517 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-85a5c0a6-426d-4fc5-98db-5155e7fd3081 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:13.463716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-85a5c0a6-426d-4fc5-98db-5155e7fd3081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:13.479688 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0bdf2-819e-4809-8971-01e63ff3ff61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:13.480398 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0bdf2-819e-4809-8971-01e63ff3ff61 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:13.480860 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0bdf2-819e-4809-8971-01e63ff3ff61 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:13.481139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0bdf2-819e-4809-8971-01e63ff3ff61 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:13.481863 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0bdf2-819e-4809-8971-01e63ff3ff61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:13.482171 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0bdf2-819e-4809-8971-01e63ff3ff61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:13.484306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0bdf2-819e-4809-8971-01e63ff3ff61 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:13.500794 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 448/912] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:43:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:13.509969 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0bdf2-819e-4809-8971-01e63ff3ff61 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:13.517969 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:13.592191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 465/913] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:43:13.592191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1cbc0-e58d-4585-a51e-2f2c30742ebc None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:13.592191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1cbc0-e58d-4585-a51e-2f2c30742ebc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:13.592191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1cbc0-e58d-4585-a51e-2f2c30742ebc None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:13.597345 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 466/914] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:13.598750 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6e2d5-280e-47d1-8ccc-638d32b2a7ca None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:13.599036 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6e2d5-280e-47d1-8ccc-638d32b2a7ca None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:13.599309 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6e2d5-280e-47d1-8ccc-638d32b2a7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:13.621521 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b0b6e2d5-280e-47d1-8ccc-638d32b2a7ca None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:13.623028 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b0b6e2d5-280e-47d1-8ccc-638d32b2a7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:13.730658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0bdf2-819e-4809-8971-01e63ff3ff61 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:13.760713 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 467/915] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:13.791479 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3bee9-29cd-497d-88cd-e05950cef2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:13.793086 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3bee9-29cd-497d-88cd-e05950cef2ab None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:13.793901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3bee9-29cd-497d-88cd-e05950cef2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:13.794726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3bee9-29cd-497d-88cd-e05950cef2ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:13.796449 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3bee9-29cd-497d-88cd-e05950cef2ab None admin] RBAC: Adding query filter params (role.id=de7f9ded8f5f43ff9ba38752e2139702, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=b11abed8d4424a1e9957c3dc9aefadd5) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:13.797268 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3bee9-29cd-497d-88cd-e05950cef2ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:13.800310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3bee9-29cd-497d-88cd-e05950cef2ab None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:13.813286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 449/916] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:43:13 2022] GET /identity/v3/role_assignments?user.id=b11abed8d4424a1e9957c3dc9aefadd5&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=de7f9ded8f5f43ff9ba38752e2139702 => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:13.898800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6e2d5-280e-47d1-8ccc-638d32b2a7ca None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:14.050443 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3bee9-29cd-497d-88cd-e05950cef2ab None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:14.061332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:14.081437 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6e2d5-280e-47d1-8ccc-638d32b2a7ca None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:14.098962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:14.138314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31631995-7d90-4bc6-acca-00501c7634aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:14.139551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31631995-7d90-4bc6-acca-00501c7634aa None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:14.140511 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31631995-7d90-4bc6-acca-00501c7634aa None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:14.140818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31631995-7d90-4bc6-acca-00501c7634aa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:14.142060 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31631995-7d90-4bc6-acca-00501c7634aa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:14.145781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31631995-7d90-4bc6-acca-00501c7634aa None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:14.164422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-31631995-7d90-4bc6-acca-00501c7634aa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:43:14.164422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:14.164422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:43:14.164422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:43:14.164422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:43:14.164422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:43:14.164422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:43:14.164422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:14.164422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:43:14.164422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:14.164422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:14.164422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:14.164422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:14.164422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:14.164422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:14.164422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:14.164422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:14.165476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:14.165476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:14.165476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:14.165476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:14.165476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:14.165476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:14.165476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:43:14.165476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:43:14.165476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:43:14.165476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:43:14.165476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:14.165476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:14.165476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:14.165476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:14.165476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:14.165476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:14.166358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:14.166358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:14.166358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:14.166358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:14.166358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:14.166358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:14.166358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:14.166358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:14.166358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:14.166358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:14.166358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:43:14.166358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:14.166358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:43:14.166358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:14.167110 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 468/917] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 24 08:43:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:14.178045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31631995-7d90-4bc6-acca-00501c7634aa None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:14.202085 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3bee9-29cd-497d-88cd-e05950cef2ab None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:14.243410 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617c97f-3a41-4195-8b58-736783ace4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:14.244557 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617c97f-3a41-4195-8b58-736783ace4dd None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:14.245094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617c97f-3a41-4195-8b58-736783ace4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:14.245594 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617c97f-3a41-4195-8b58-736783ace4dd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:14.246452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e617c97f-3a41-4195-8b58-736783ace4dd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:14.248055 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e617c97f-3a41-4195-8b58-736783ace4dd None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:14.258401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-e617c97f-3a41-4195-8b58-736783ace4dd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:43:14.258401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:14.258401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:43:14.258401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:43:14.258401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:43:14.258401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:43:14.258401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:43:14.258401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:14.258401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:43:14.258401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:14.258401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:14.258401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:14.258401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:14.258401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:14.258401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:14.258401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:14.258401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:14.259657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:14.259657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:14.259657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:14.259657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:14.259657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:14.259657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:14.259657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:43:14.259657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:43:14.259657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:43:14.259657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:43:14.259657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:14.259657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:14.259657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:14.259657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:14.259657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:14.259657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:14.260709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:14.260709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:14.260709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:14.260709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:14.260709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:14.260709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:14.260709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:14.260709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:14.260709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:14.260709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:14.260709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:43:14.260709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:14.260709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 08:43:14.260709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:14.264207 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 450/918] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:43:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:43:14.267282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4af06-65fb-4c0c-9e24-67ea7b412396 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:14.267851 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4af06-65fb-4c0c-9e24-67ea7b412396 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:14.268444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4af06-65fb-4c0c-9e24-67ea7b412396 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:14.271048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 451/919] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:14.278539 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4af06-65fb-4c0c-9e24-67ea7b412396 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:14.342558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31631995-7d90-4bc6-acca-00501c7634aa None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:14.386490 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f22d80-454a-4427-88d2-c6ee655e89a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:14.387106 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f22d80-454a-4427-88d2-c6ee655e89a0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:14.387661 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f22d80-454a-4427-88d2-c6ee655e89a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:14.387970 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f22d80-454a-4427-88d2-c6ee655e89a0 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:14.388769 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f22d80-454a-4427-88d2-c6ee655e89a0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:14.389068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f22d80-454a-4427-88d2-c6ee655e89a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:14.391656 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f22d80-454a-4427-88d2-c6ee655e89a0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:14.400138 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 469/920] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 24 08:43:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:14.401577 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabb308-d7ef-486b-843e-34fef89e8969 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:14.401862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabb308-d7ef-486b-843e-34fef89e8969 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:14.402130 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabb308-d7ef-486b-843e-34fef89e8969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:14.409769 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:14.422164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2fabb308-d7ef-486b-843e-34fef89e8969 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:14.423369 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2fabb308-d7ef-486b-843e-34fef89e8969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:14.430725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4af06-65fb-4c0c-9e24-67ea7b412396 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:14.437080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:14.472154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981a840-c7ee-4d63-826f-15a9015d5c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:14.472807 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981a840-c7ee-4d63-826f-15a9015d5c65 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:14.473080 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981a840-c7ee-4d63-826f-15a9015d5c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:14.473345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981a840-c7ee-4d63-826f-15a9015d5c65 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:14.474027 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981a840-c7ee-4d63-826f-15a9015d5c65 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:14.474300 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981a840-c7ee-4d63-826f-15a9015d5c65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:14.475749 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981a840-c7ee-4d63-826f-15a9015d5c65 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:14.486951 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 452/921] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:43:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:14.486951 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981a840-c7ee-4d63-826f-15a9015d5c65 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:14.520164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 470/922] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:14.520164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabb308-d7ef-486b-843e-34fef89e8969 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:14.608217 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981a840-c7ee-4d63-826f-15a9015d5c65 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:14.647270 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98130c1d-dd39-46df-9b96-708ac44fde74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:14.648525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98130c1d-dd39-46df-9b96-708ac44fde74 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:14.649102 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98130c1d-dd39-46df-9b96-708ac44fde74 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:14.649641 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98130c1d-dd39-46df-9b96-708ac44fde74 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:14.650487 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98130c1d-dd39-46df-9b96-708ac44fde74 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:14.651021 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98130c1d-dd39-46df-9b96-708ac44fde74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:14.652720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98130c1d-dd39-46df-9b96-708ac44fde74 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:14.660886 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 453/923] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 24 08:43:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:14.672041 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98130c1d-dd39-46df-9b96-708ac44fde74 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:14.696429 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:14.700858 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabb308-d7ef-486b-843e-34fef89e8969 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:14.777291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a9e43-b8f6-461a-a9c7-c4b46b52cde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:14.778335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a9e43-b8f6-461a-a9c7-c4b46b52cde5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:14.778832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a9e43-b8f6-461a-a9c7-c4b46b52cde5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:14.780073 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a9e43-b8f6-461a-a9c7-c4b46b52cde5 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:14.781351 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a9e43-b8f6-461a-a9c7-c4b46b52cde5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:14.781917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a9e43-b8f6-461a-a9c7-c4b46b52cde5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:14.786046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a9e43-b8f6-461a-a9c7-c4b46b52cde5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:14.798803 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 471/924] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 24 08:43:14 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:14.800451 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a9e43-b8f6-461a-a9c7-c4b46b52cde5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:14.812666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98130c1d-dd39-46df-9b96-708ac44fde74 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:14.825335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:14.869364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b27c7b-da53-42be-8643-8823eaa40726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:14.870230 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b27c7b-da53-42be-8643-8823eaa40726 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:14.870677 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b27c7b-da53-42be-8643-8823eaa40726 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:14.870985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b27c7b-da53-42be-8643-8823eaa40726 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:14.871815 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b27c7b-da53-42be-8643-8823eaa40726 None admin] RBAC: Adding query filter params (interface=public, service_id=1b514de88e5c414ea53ba7f49864d482, region_id=RegionOne) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:14.872134 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b27c7b-da53-42be-8643-8823eaa40726 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:14.874556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b27c7b-da53-42be-8643-8823eaa40726 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:14.891482 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 454/925] 10.222.0.67 () {56 vars in 1251 bytes} [Wed Aug 24 08:43:14 2022] GET /identity/v3/endpoints?service_id=1b514de88e5c414ea53ba7f49864d482&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:14.891482 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b27c7b-da53-42be-8643-8823eaa40726 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:14.945867 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a9e43-b8f6-461a-a9c7-c4b46b52cde5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:15.001434 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc981320-01e1-4380-832a-78aba7ff42cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:15.002477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc981320-01e1-4380-832a-78aba7ff42cb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:15.002955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc981320-01e1-4380-832a-78aba7ff42cb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:15.003430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc981320-01e1-4380-832a-78aba7ff42cb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:15.004509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc981320-01e1-4380-832a-78aba7ff42cb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:15.007099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc981320-01e1-4380-832a-78aba7ff42cb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:15.012023 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:15.025944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-bc981320-01e1-4380-832a-78aba7ff42cb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:43:15.025944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:15.025944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:43:15.025944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:43:15.025944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:43:15.025944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:43:15.025944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:43:15.025944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:15.025944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:43:15.025944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:15.025944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:15.025944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:15.025944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:15.025944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:15.025944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:15.025944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:15.027999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:15.027999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:15.027999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:15.027999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:15.027999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:15.027999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:15.027999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:15.027999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:43:15.027999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:43:15.027999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:43:15.027999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:43:15.027999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:15.027999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:15.027999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:15.027999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:15.027999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:15.027999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:15.029930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:15.029930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:15.029930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:15.029930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:15.029930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:15.029930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:15.029930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:15.029930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:15.029930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:15.029930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:15.029930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:43:15.029930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:15.029930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 08:43:15.029930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:15.029930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 472/926] 10.222.0.67 () {56 vars in 1132 bytes} [Wed Aug 24 08:43:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:15.031472 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc981320-01e1-4380-832a-78aba7ff42cb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:15.064904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b27c7b-da53-42be-8643-8823eaa40726 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:15.127363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0322088-63c6-4ee6-b90f-15264533b671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:15.128295 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0322088-63c6-4ee6-b90f-15264533b671 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:15.128761 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0322088-63c6-4ee6-b90f-15264533b671 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:15.129180 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0322088-63c6-4ee6-b90f-15264533b671 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:15.130318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0322088-63c6-4ee6-b90f-15264533b671 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:15.133190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0322088-63c6-4ee6-b90f-15264533b671 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:15.137401 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:15.180172 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 455/927] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:43:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:43:15.180172 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0322088-63c6-4ee6-b90f-15264533b671 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:15.199860 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc981320-01e1-4380-832a-78aba7ff42cb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:15.274950 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423cac6-8e33-471e-a10c-82eb89609703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:15.280128 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423cac6-8e33-471e-a10c-82eb89609703 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:15.280128 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423cac6-8e33-471e-a10c-82eb89609703 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:15.280128 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423cac6-8e33-471e-a10c-82eb89609703 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:15.280128 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423cac6-8e33-471e-a10c-82eb89609703 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:15.284133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423cac6-8e33-471e-a10c-82eb89609703 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:15.296147 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 473/928] 10.222.0.67 () {56 vars in 1087 bytes} [Wed Aug 24 08:43:15 2022] GET /identity/v3/services => generated 1497 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:43:15.382289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0322088-63c6-4ee6-b90f-15264533b671 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:15.450632 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1e1fa-d8e2-4fe5-a9db-a2466a3d3687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:15.451655 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1e1fa-d8e2-4fe5-a9db-a2466a3d3687 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:15.452212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1e1fa-d8e2-4fe5-a9db-a2466a3d3687 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:15.452712 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1e1fa-d8e2-4fe5-a9db-a2466a3d3687 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:15.453870 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1e1fa-d8e2-4fe5-a9db-a2466a3d3687 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:15.454386 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1e1fa-d8e2-4fe5-a9db-a2466a3d3687 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:15.456926 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1e1fa-d8e2-4fe5-a9db-a2466a3d3687 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:15.459165 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:15.470268 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 456/929] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 24 08:43:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:15.477817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423cac6-8e33-471e-a10c-82eb89609703 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:15.638741 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423cac6-8e33-471e-a10c-82eb89609703 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:15.702822 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b009a-c410-4fbd-8891-929ecd9988cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:15.703898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b009a-c410-4fbd-8891-929ecd9988cf None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:15.704411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b009a-c410-4fbd-8891-929ecd9988cf None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:15.704920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b009a-c410-4fbd-8891-929ecd9988cf None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:15.705912 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b009a-c410-4fbd-8891-929ecd9988cf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:15.706468 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b009a-c410-4fbd-8891-929ecd9988cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:15.709310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b009a-c410-4fbd-8891-929ecd9988cf None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:15.712886 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:15.726725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 474/930] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 24 08:43:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:15.732005 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1e1fa-d8e2-4fe5-a9db-a2466a3d3687 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:15.905420 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1e1fa-d8e2-4fe5-a9db-a2466a3d3687 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:15.948383 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15c0f5-79e9-4578-8810-472694d31ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:15.949003 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15c0f5-79e9-4578-8810-472694d31ed6 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:15.949299 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15c0f5-79e9-4578-8810-472694d31ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:15.949567 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15c0f5-79e9-4578-8810-472694d31ed6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:15.950209 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15c0f5-79e9-4578-8810-472694d31ed6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:15.951584 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15c0f5-79e9-4578-8810-472694d31ed6 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:15.978097 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 457/931] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:43:15 2022] POST /identity/v3/endpoints => generated 347 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:43:16.207828 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7fb27-a1e0-4aee-94f2-19d75f166b37 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:16.208088 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7fb27-a1e0-4aee-94f2-19d75f166b37 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:16.208412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7fb27-a1e0-4aee-94f2-19d75f166b37 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:16.209977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 475/932] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:16.219442 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a52ef-0e5c-4ad5-b30a-c1ccc393f80b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:16.220890 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a52ef-0e5c-4ad5-b30a-c1ccc393f80b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:16.221664 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a52ef-0e5c-4ad5-b30a-c1ccc393f80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:16.252342 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-834a52ef-0e5c-4ad5-b30a-c1ccc393f80b None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:16.254831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-834a52ef-0e5c-4ad5-b30a-c1ccc393f80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:16.342676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:16.376594 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 458/933] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:43:16.892279 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7fb27-a1e0-4aee-94f2-19d75f166b37 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:17.004638 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7fb27-a1e0-4aee-94f2-19d75f166b37 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:17.041538 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f8335-77ef-4b3b-8c5c-3849cc6ff6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:17.042187 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f8335-77ef-4b3b-8c5c-3849cc6ff6ef None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:17.042468 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f8335-77ef-4b3b-8c5c-3849cc6ff6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:17.042740 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f8335-77ef-4b3b-8c5c-3849cc6ff6ef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:17.054722 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f8335-77ef-4b3b-8c5c-3849cc6ff6ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:43:17.059408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ea4f8335-77ef-4b3b-8c5c-3849cc6ff6ef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:43:17.059408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:17.059408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:17.059408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:17.059408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:17.059408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:17.059408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:17.059408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:17.059408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:17.059408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:17.059408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:17.059408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:17.059408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:43:17.059408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:43:17.059408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:43:17.059408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:43:17.060374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:43:17.060374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:43:17.060374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:43:17.060374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:43:17.060374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:17.060374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:17.060374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:17.060374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:17.060374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:17.060374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:17.060374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:17.060374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:17.060374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:17.060374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:17.060374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:17.060374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:17.061234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:17.061234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:17.061234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:17.061234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:17.061234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:43:17.061234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:43:17.061234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:43:17.061234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:43:17.061234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:43:17.061234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:43:17.061234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:43:17.061234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:17.061234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 476/934] 10.222.0.67 () {56 vars in 1120 bytes} [Wed Aug 24 08:43:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:43:17.068475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a52ef-0e5c-4ad5-b30a-c1ccc393f80b None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:17.177291 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a52ef-0e5c-4ad5-b30a-c1ccc393f80b None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:17.216138 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18969247-de2c-4ad7-ab82-4b655b0fce09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:17.217203 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18969247-de2c-4ad7-ab82-4b655b0fce09 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:17.217742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18969247-de2c-4ad7-ab82-4b655b0fce09 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:17.218218 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18969247-de2c-4ad7-ab82-4b655b0fce09 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:17.219116 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18969247-de2c-4ad7-ab82-4b655b0fce09 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:17.219634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18969247-de2c-4ad7-ab82-4b655b0fce09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:17.221795 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18969247-de2c-4ad7-ab82-4b655b0fce09 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:17.232860 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 459/935] 10.222.0.67 () {56 vars in 1115 bytes} [Wed Aug 24 08:43:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:17.238907 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f8335-77ef-4b3b-8c5c-3849cc6ff6ef None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:17.302980 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:17.347389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f8335-77ef-4b3b-8c5c-3849cc6ff6ef None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:17.384027 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8dd3ea-5b19-43f4-b25e-17f662115a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:17.384702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8dd3ea-5b19-43f4-b25e-17f662115a1c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:17.384985 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8dd3ea-5b19-43f4-b25e-17f662115a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:17.385257 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8dd3ea-5b19-43f4-b25e-17f662115a1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:17.396389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8dd3ea-5b19-43f4-b25e-17f662115a1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:17.398759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8dd3ea-5b19-43f4-b25e-17f662115a1c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:17.414214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-1a8dd3ea-5b19-43f4-b25e-17f662115a1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:17.414214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:17.414214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:17.414214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:17.414214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:17.414214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:17.414214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:17.414214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:17.414214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:17.414214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:17.414214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:17.414214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:17.414214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:43:17.414214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:43:17.414214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:43:17.414214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:43:17.415301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:17.415301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:17.415301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:17.415301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:17.415301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:17.415301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:17.415301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:17.415301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:17.415301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:17.415301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:17.415301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:17.415301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:17.415301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:17.415301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:17.415301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:17.415301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:17.415301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:43:17.416247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:43:17.416247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:17.416247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:17.416247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 477/936] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:17.423195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18969247-de2c-4ad7-ab82-4b655b0fce09 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:17.503533 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:17.515357 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe8e9c-13c1-4b7d-8106-4cd374b31f7b None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:17.515680 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe8e9c-13c1-4b7d-8106-4cd374b31f7b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:17.515918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe8e9c-13c1-4b7d-8106-4cd374b31f7b None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:17.517612 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 478/937] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:43:17.523446 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a878d-d213-4e6c-8bec-ebe641fa67ce None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:17.523838 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a878d-d213-4e6c-8bec-ebe641fa67ce None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:17.524048 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a878d-d213-4e6c-8bec-ebe641fa67ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:17.543405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-070a878d-d213-4e6c-8bec-ebe641fa67ce None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:17.544848 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-070a878d-d213-4e6c-8bec-ebe641fa67ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:17.578897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18969247-de2c-4ad7-ab82-4b655b0fce09 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:17.619150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc9344-b263-47ac-a72a-971936cbc4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:17.620211 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc9344-b263-47ac-a72a-971936cbc4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:17.620694 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc9344-b263-47ac-a72a-971936cbc4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:17.621178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc9344-b263-47ac-a72a-971936cbc4d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:17.621696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 479/938] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:43:17.622267 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cc9344-b263-47ac-a72a-971936cbc4d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:17.622765 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cc9344-b263-47ac-a72a-971936cbc4d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:17.624461 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cc9344-b263-47ac-a72a-971936cbc4d5 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:17.640485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 460/939] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:17.649828 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a878d-d213-4e6c-8bec-ebe641fa67ce None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:17.729616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:17.847890 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a878d-d213-4e6c-8bec-ebe641fa67ce None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:17.905344 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb882afc-93ad-45f7-b70a-73092b2b6bf3 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:17.905672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb882afc-93ad-45f7-b70a-73092b2b6bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:17.905888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb882afc-93ad-45f7-b70a-73092b2b6bf3 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:17.907802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 461/940] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:17.913436 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14405ee-83f7-471b-ae8a-6ac09207e273 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:17.913865 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14405ee-83f7-471b-ae8a-6ac09207e273 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:17.914330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14405ee-83f7-471b-ae8a-6ac09207e273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:17.926691 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d866a-8e00-4aec-85c2-f2bfb21615d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:17.927750 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d866a-8e00-4aec-85c2-f2bfb21615d8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:17.928027 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d866a-8e00-4aec-85c2-f2bfb21615d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:17.928327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d866a-8e00-4aec-85c2-f2bfb21615d8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:17.940047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d866a-8e00-4aec-85c2-f2bfb21615d8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:17.942009 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d866a-8e00-4aec-85c2-f2bfb21615d8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:17.948406 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-a14405ee-83f7-471b-ae8a-6ac09207e273 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:17.950395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a14405ee-83f7-471b-ae8a-6ac09207e273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:17.962895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-1a4d866a-8e00-4aec-85c2-f2bfb21615d8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:43:17.962895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:17.962895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:17.962895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:17.962895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:17.962895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:17.962895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:17.962895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:17.962895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:17.962895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:17.962895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:17.962895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:17.962895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:43:17.962895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:43:17.962895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:43:17.962895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:43:17.963891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:17.963891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:17.963891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:43:17.963891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:17.963891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:43:17.963891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:17.963891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:17.963891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:17.963891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:17.963891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:17.963891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:17.963891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:17.963891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:17.963891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:17.963891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:17.963891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:17.963891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:17.963891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:17.964873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:17.964873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:17.964873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:43:17.964873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:43:17.964873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:43:17.964873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:43:17.964873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:43:17.964873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:43:17.964873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:43:17.964873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:17.966074 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 480/941] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 24 08:43:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:17.972889 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d866a-8e00-4aec-85c2-f2bfb21615d8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:18.077472 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 462/942] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:18.121631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14405ee-83f7-471b-ae8a-6ac09207e273 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:18.156971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:18.160671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d866a-8e00-4aec-85c2-f2bfb21615d8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:18.168193 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:18.201385 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bdf56-d558-4275-bbb6-7faa36152561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:18.202104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bdf56-d558-4275-bbb6-7faa36152561 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:18.202385 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bdf56-d558-4275-bbb6-7faa36152561 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:18.202659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bdf56-d558-4275-bbb6-7faa36152561 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:18.203411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084bdf56-d558-4275-bbb6-7faa36152561 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:18.203717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084bdf56-d558-4275-bbb6-7faa36152561 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:18.205534 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084bdf56-d558-4275-bbb6-7faa36152561 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:18.230632 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 481/943] 10.222.0.67 () {56 vars in 1149 bytes} [Wed Aug 24 08:43:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:18.232289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386ea94-2d2f-450b-985e-098bc20620c0 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:18.232579 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386ea94-2d2f-450b-985e-098bc20620c0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:18.232964 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386ea94-2d2f-450b-985e-098bc20620c0 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:18.234423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 482/944] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:18.240046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386ea94-2d2f-450b-985e-098bc20620c0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:18.267736 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14405ee-83f7-471b-ae8a-6ac09207e273 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:18.311956 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ba75a-6ab6-485b-ab36-c1805fd70fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:18.312779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ba75a-6ab6-485b-ab36-c1805fd70fca None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:18.313167 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ba75a-6ab6-485b-ab36-c1805fd70fca None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:18.313454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ba75a-6ab6-485b-ab36-c1805fd70fca None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:18.314168 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ba75a-6ab6-485b-ab36-c1805fd70fca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:18.315620 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ba75a-6ab6-485b-ab36-c1805fd70fca None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:18.320203 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: WARNING keystone.server.flask.common [None req-8d3ba75a-6ab6-485b-ab36-c1805fd70fca None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 08:43:18.344943 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.sql.core [None req-8d3ba75a-6ab6-485b-ab36-c1805fd70fca None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 08:43:18.344943 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 08:43:18.344943 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [parameters: {'id': '301e1f4f558444c888b13c3ece2479da', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 08:43:18.344943 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81476) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 08:43:18.368798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-8d3ba75a-6ab6-485b-ab36-c1805fd70fca None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 301e1f4f558444c888b13c3ece2479da.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 301e1f4f558444c888b13c3ece2479da. Aug 24 08:43:18.368798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:18.368798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 08:43:18.368798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 08:43:18.368798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 08:43:18.368798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 08:43:18.368798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 08:43:18.368798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 08:43:18.368798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 08:43:18.368798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application conn.query(q) Aug 24 08:43:18.368798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 08:43:18.368798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 08:43:18.368798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 08:43:18.369839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application result.read() Aug 24 08:43:18.369839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 08:43:18.369839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 08:43:18.369839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 08:43:18.369839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 08:43:18.369839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 08:43:18.369839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 08:43:18.369839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 08:43:18.369839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 08:43:18.369839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 08:43:18.369839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:18.369839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 08:43:18.369839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:18.369839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:18.369839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 08:43:18.369839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 08:43:18.369839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 08:43:18.369839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 08:43:18.371944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 08:43:18.371944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application next(self.gen) Aug 24 08:43:18.371944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 08:43:18.371944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application yield resource Aug 24 08:43:18.371944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 08:43:18.371944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application next(self.gen) Aug 24 08:43:18.371944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 08:43:18.371944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 08:43:18.371944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 08:43:18.371944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 08:43:18.371944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 08:43:18.371944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise self.value Aug 24 08:43:18.371944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 08:43:18.371944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 08:43:18.371944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 08:43:18.371944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application session.commit() Aug 24 08:43:18.371944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 08:43:18.373087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 08:43:18.373087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 08:43:18.373087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 08:43:18.373087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 08:43:18.373087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self.session.flush() Aug 24 08:43:18.373087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 08:43:18.373087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 08:43:18.373087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 08:43:18.373087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 08:43:18.373087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 08:43:18.373087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application compat.raise_( Aug 24 08:43:18.373087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 08:43:18.373087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception Aug 24 08:43:18.373087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 08:43:18.373087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 08:43:18.373087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 08:43:18.373087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 08:43:18.374163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 08:43:18.374163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 08:43:18.374163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 08:43:18.374163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 08:43:18.374163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 08:43:18.374163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 08:43:18.374163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 08:43:18.374163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 08:43:18.374163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 08:43:18.374163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 08:43:18.374163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 08:43:18.374163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 08:43:18.374163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 08:43:18.374163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 08:43:18.374163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 08:43:18.375111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 08:43:18.375111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 08:43:18.375111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception Aug 24 08:43:18.375111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 08:43:18.375111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 08:43:18.375111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 08:43:18.375111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 08:43:18.375111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 08:43:18.375111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 08:43:18.375111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 08:43:18.375111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application conn.query(q) Aug 24 08:43:18.375111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 08:43:18.375111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 08:43:18.375111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 08:43:18.375111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application result.read() Aug 24 08:43:18.375111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 08:43:18.375111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 08:43:18.376121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 08:43:18.376121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 08:43:18.376121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 08:43:18.376121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 08:43:18.376121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 08:43:18.376121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 08:43:18.376121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 08:43:18.376121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 08:43:18.376121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [parameters: {'id': '301e1f4f558444c888b13c3ece2479da', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 08:43:18.376121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 08:43:18.376121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:18.376121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:18.376121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:18.376121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:18.376121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 08:43:18.377111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 08:43:18.377111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 08:43:18.377111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 08:43:18.377111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 08:43:18.377111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:18.377111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:18.377111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:18.377111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:18.377111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:18.377111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:18.377111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:18.377111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:18.377111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:18.377111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:18.377111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:18.377111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:18.377111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:18.378226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:18.378226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 08:43:18.378226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 08:43:18.378226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:18.378226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:18.378226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 08:43:18.378226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 08:43:18.378226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 301e1f4f558444c888b13c3ece2479da. Aug 24 08:43:18.378226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:18.378226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 463/945] 10.222.0.67 () {60 vars in 1140 bytes} [Wed Aug 24 08:43:18 2022] POST /identity/v3/projects => generated 262 bytes in 250 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:43:18.378226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840e614-63bf-4995-9b08-3980b0d20924 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:18.378226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840e614-63bf-4995-9b08-3980b0d20924 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:18.379013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840e614-63bf-4995-9b08-3980b0d20924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:18.382618 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386ea94-2d2f-450b-985e-098bc20620c0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:18.397003 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d840e614-63bf-4995-9b08-3980b0d20924 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:18.398669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d840e614-63bf-4995-9b08-3980b0d20924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:18.438394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524230d-667b-439d-b67f-b8a32804997a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:18.439361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524230d-667b-439d-b67f-b8a32804997a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:18.439821 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524230d-667b-439d-b67f-b8a32804997a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:18.440306 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524230d-667b-439d-b67f-b8a32804997a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:18.459013 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e524230d-667b-439d-b67f-b8a32804997a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:18.462261 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e524230d-667b-439d-b67f-b8a32804997a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:18.471255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 464/946] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:18.472130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840e614-63bf-4995-9b08-3980b0d20924 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:18.480134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-e524230d-667b-439d-b67f-b8a32804997a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:18.480134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:18.480134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:18.480134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:18.480134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:18.480134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:18.480134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:18.480134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:18.480134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:18.480134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:18.480134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:18.480134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:18.480134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:43:18.480134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:43:18.480134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:43:18.480134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:43:18.488374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:18.488374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:18.488374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:18.488374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:18.488374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:18.488374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:18.488374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:18.488374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:18.488374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:18.488374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:18.488374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:18.488374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:18.488374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:18.488374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:18.488374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:18.488374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:18.488374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:43:18.490575 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:43:18.490575 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:18.490575 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:18.490575 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 483/947] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:18.490575 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524230d-667b-439d-b67f-b8a32804997a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:18.508049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:18.519872 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:18.626028 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524230d-667b-439d-b67f-b8a32804997a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:18.665437 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81adb7e6-e715-4400-b7f8-56f5a6ebf03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:18.666081 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adb7e6-e715-4400-b7f8-56f5a6ebf03e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:18.666360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adb7e6-e715-4400-b7f8-56f5a6ebf03e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:18.666638 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adb7e6-e715-4400-b7f8-56f5a6ebf03e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:18.666917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840e614-63bf-4995-9b08-3980b0d20924 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:18.674136 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81adb7e6-e715-4400-b7f8-56f5a6ebf03e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:18.675600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81adb7e6-e715-4400-b7f8-56f5a6ebf03e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:18.688898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-81adb7e6-e715-4400-b7f8-56f5a6ebf03e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:18.688898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:18.688898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:18.688898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:18.688898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:18.688898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:18.688898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:18.688898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:18.688898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:18.688898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:18.688898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:18.688898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:18.688898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:43:18.688898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:43:18.688898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:43:18.690090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:43:18.690090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:18.690090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:18.690090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:18.690090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:18.690090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:18.690090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:18.690090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:18.690090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:18.690090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:18.690090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:18.690090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:18.690090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:18.690090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:18.690090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:18.690090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:18.690090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:18.691034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:43:18.691034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:43:18.691034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:43:18.691034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:43:18.691034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:43:18.691034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:43:18.691034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:18.691034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:18.691034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 484/948] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:43:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:18.691034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81adb7e6-e715-4400-b7f8-56f5a6ebf03e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:18.728504 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8663b55-e00b-4d51-92c5-e8f312308228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:18.729157 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8663b55-e00b-4d51-92c5-e8f312308228 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:18.729432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8663b55-e00b-4d51-92c5-e8f312308228 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:18.729705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8663b55-e00b-4d51-92c5-e8f312308228 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:18.730298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8663b55-e00b-4d51-92c5-e8f312308228 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:18.731885 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8663b55-e00b-4d51-92c5-e8f312308228 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:18.743249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-d8663b55-e00b-4d51-92c5-e8f312308228 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:43:18.743249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:18.743249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:43:18.743249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:43:18.743249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:43:18.743249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:43:18.743249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:43:18.743249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:18.743249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:43:18.743249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:18.743249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:18.743249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:18.743249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:18.743249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:18.743249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:18.743249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:18.743249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:18.744452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:18.744452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:18.744452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:18.744452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:18.744452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:18.744452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:18.744452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:43:18.744452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:43:18.744452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:43:18.744452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:43:18.744452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:18.744452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:18.744452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:18.744452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:18.744452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:18.744452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:18.745306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:18.745306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:18.745306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:18.745306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:18.745306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:18.745306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:18.745306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:18.745306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:18.745306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:18.745306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:18.745306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:43:18.745306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:18.745306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:43:18.745306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:18.745306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 465/949] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 24 08:43:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:18.746113 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8663b55-e00b-4d51-92c5-e8f312308228 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:18.790267 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:18.843412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81adb7e6-e715-4400-b7f8-56f5a6ebf03e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:18.854214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:18.879533 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8663b55-e00b-4d51-92c5-e8f312308228 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:18.882914 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36a4cc-8207-4ee8-b056-b31b8b9518d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:18.883549 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36a4cc-8207-4ee8-b056-b31b8b9518d5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:18.883827 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36a4cc-8207-4ee8-b056-b31b8b9518d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:18.884116 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36a4cc-8207-4ee8-b056-b31b8b9518d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:18.884773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee36a4cc-8207-4ee8-b056-b31b8b9518d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:18.885077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee36a4cc-8207-4ee8-b056-b31b8b9518d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:18.886453 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee36a4cc-8207-4ee8-b056-b31b8b9518d5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:18.899667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 485/950] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:18.900559 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36a4cc-8207-4ee8-b056-b31b8b9518d5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:18.938894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd6e84-320d-4630-9f83-508d2e757190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:18.939703 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd6e84-320d-4630-9f83-508d2e757190 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:18.940254 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd6e84-320d-4630-9f83-508d2e757190 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:18.940747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd6e84-320d-4630-9f83-508d2e757190 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:18.941947 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd6e84-320d-4630-9f83-508d2e757190 None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:18.942498 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd6e84-320d-4630-9f83-508d2e757190 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:18.945884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd6e84-320d-4630-9f83-508d2e757190 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:18.980358 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 466/951] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 24 08:43:18 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:18.981221 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd6e84-320d-4630-9f83-508d2e757190 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:19.048283 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36a4cc-8207-4ee8-b056-b31b8b9518d5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:19.087421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:19.099266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b9831-f2fc-4a37-b770-aa0be1bb0d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:19.100057 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b9831-f2fc-4a37-b770-aa0be1bb0d3a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:19.100448 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b9831-f2fc-4a37-b770-aa0be1bb0d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:19.101277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b9831-f2fc-4a37-b770-aa0be1bb0d3a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:19.102078 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b9831-f2fc-4a37-b770-aa0be1bb0d3a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:19.103507 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b9831-f2fc-4a37-b770-aa0be1bb0d3a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:19.113573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-a27b9831-f2fc-4a37-b770-aa0be1bb0d3a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:43:19.113573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:19.113573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:43:19.113573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:43:19.113573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:43:19.113573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:43:19.113573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:43:19.113573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:19.113573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:43:19.113573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:19.113573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:19.113573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:19.113573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:19.113573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:19.113573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:19.113573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:19.113573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:19.115454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:19.115454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:19.115454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:19.115454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:19.115454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:19.115454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:19.115454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:43:19.115454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:43:19.115454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:43:19.115454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:43:19.115454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:19.115454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:19.115454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:19.115454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:19.115454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:19.115454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:19.119018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:19.119018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:19.119018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:19.119018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:19.119018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:19.119018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:19.119018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:19.119018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:19.119018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:19.119018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:19.119018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:43:19.119018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:19.119018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:43:19.119018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:19.119018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 486/952] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:43:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:19.122552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b9831-f2fc-4a37-b770-aa0be1bb0d3a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:19.139001 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd6e84-320d-4630-9f83-508d2e757190 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:19.200999 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91121f10-a02a-4104-bb00-48889e01cb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:19.202554 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91121f10-a02a-4104-bb00-48889e01cb28 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:19.202554 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91121f10-a02a-4104-bb00-48889e01cb28 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:19.202955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91121f10-a02a-4104-bb00-48889e01cb28 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:19.204343 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91121f10-a02a-4104-bb00-48889e01cb28 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:19.204485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91121f10-a02a-4104-bb00-48889e01cb28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:19.206825 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91121f10-a02a-4104-bb00-48889e01cb28 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:19.219206 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 467/953] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 24 08:43:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:19.219206 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91121f10-a02a-4104-bb00-48889e01cb28 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:19.232256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:19.260872 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b9831-f2fc-4a37-b770-aa0be1bb0d3a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:19.316064 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d398ed-c864-461e-a30e-f3854c823b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:19.316797 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d398ed-c864-461e-a30e-f3854c823b9e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:19.317239 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d398ed-c864-461e-a30e-f3854c823b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:19.317580 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d398ed-c864-461e-a30e-f3854c823b9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:19.327338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d398ed-c864-461e-a30e-f3854c823b9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:19.329210 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d398ed-c864-461e-a30e-f3854c823b9e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:19.341414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-90d398ed-c864-461e-a30e-f3854c823b9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:19.341414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:19.341414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:19.341414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:19.341414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:19.341414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:19.341414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:19.341414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:19.341414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:19.341414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:19.341414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:19.341414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:19.341414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:43:19.341414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:43:19.341414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:43:19.343204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:43:19.343204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:19.343204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:19.343204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:19.343204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:19.343204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:19.343204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:19.343204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:19.343204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:19.343204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:19.343204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:19.343204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:19.343204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:19.343204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:19.343204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:19.343204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:19.343204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:19.345141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:43:19.345141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:43:19.345141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:43:19.345141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:43:19.345141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:43:19.345141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:43:19.345141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:19.345141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:19.345141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 487/954] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:43:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:19.345141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d398ed-c864-461e-a30e-f3854c823b9e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:19.372641 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91121f10-a02a-4104-bb00-48889e01cb28 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:19.421101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375720dd-eb8b-4e89-b598-b1ba99fbe76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:19.422754 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375720dd-eb8b-4e89-b598-b1ba99fbe76b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:19.423478 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375720dd-eb8b-4e89-b598-b1ba99fbe76b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:19.424158 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375720dd-eb8b-4e89-b598-b1ba99fbe76b None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:19.425168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:19.425583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375720dd-eb8b-4e89-b598-b1ba99fbe76b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:19.426577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375720dd-eb8b-4e89-b598-b1ba99fbe76b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:19.431739 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375720dd-eb8b-4e89-b598-b1ba99fbe76b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:19.441596 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 468/955] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 24 08:43:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:19.442445 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375720dd-eb8b-4e89-b598-b1ba99fbe76b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:19.484996 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d398ed-c864-461e-a30e-f3854c823b9e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:19.533975 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001bd5e3-38e5-4cb6-a33a-8092cce56a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:19.535040 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001bd5e3-38e5-4cb6-a33a-8092cce56a74 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:19.535561 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001bd5e3-38e5-4cb6-a33a-8092cce56a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:19.536072 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001bd5e3-38e5-4cb6-a33a-8092cce56a74 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:19.537325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001bd5e3-38e5-4cb6-a33a-8092cce56a74 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:19.537897 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001bd5e3-38e5-4cb6-a33a-8092cce56a74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:19.540715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001bd5e3-38e5-4cb6-a33a-8092cce56a74 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:19.551786 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 488/956] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 24 08:43:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:19.553590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001bd5e3-38e5-4cb6-a33a-8092cce56a74 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:19.567624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:19.616953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375720dd-eb8b-4e89-b598-b1ba99fbe76b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:19.686335 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e0470-7997-48e4-bc21-1b10f2be0fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:19.687069 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e0470-7997-48e4-bc21-1b10f2be0fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:19.687409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e0470-7997-48e4-bc21-1b10f2be0fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:19.687686 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e0470-7997-48e4-bc21-1b10f2be0fd4 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:19.688490 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e0470-7997-48e4-bc21-1b10f2be0fd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:19.688800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e0470-7997-48e4-bc21-1b10f2be0fd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:19.690469 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e0470-7997-48e4-bc21-1b10f2be0fd4 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:19.693267 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001bd5e3-38e5-4cb6-a33a-8092cce56a74 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:19.703702 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 469/957] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:43:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:19.704748 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e0470-7997-48e4-bc21-1b10f2be0fd4 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:19.731013 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:19.765356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d0db3-9c80-4ff8-94f6-cc805f73b878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:19.766719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d0db3-9c80-4ff8-94f6-cc805f73b878 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:19.767473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d0db3-9c80-4ff8-94f6-cc805f73b878 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:19.768206 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d0db3-9c80-4ff8-94f6-cc805f73b878 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:19.769367 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d0db3-9c80-4ff8-94f6-cc805f73b878 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:19.769888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d0db3-9c80-4ff8-94f6-cc805f73b878 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:19.772667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d0db3-9c80-4ff8-94f6-cc805f73b878 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:19.792511 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 489/958] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 24 08:43:19 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:19.793627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d0db3-9c80-4ff8-94f6-cc805f73b878 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:19.875187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e0470-7997-48e4-bc21-1b10f2be0fd4 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:19.937726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:19.947357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e002e0a-faab-4a49-b30e-acdd597f23d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:19.948391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e002e0a-faab-4a49-b30e-acdd597f23d3 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:19.949543 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e002e0a-faab-4a49-b30e-acdd597f23d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:19.950082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e002e0a-faab-4a49-b30e-acdd597f23d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:19.951264 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e002e0a-faab-4a49-b30e-acdd597f23d3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:19.953260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e002e0a-faab-4a49-b30e-acdd597f23d3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:19.965892 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d0db3-9c80-4ff8-94f6-cc805f73b878 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:19.980712 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 470/959] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:43:19 2022] POST /identity/v3/endpoints => generated 349 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:43:20.049955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39635856-549c-4cab-aa55-233542fbb83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:20.051029 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39635856-549c-4cab-aa55-233542fbb83b None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:20.051522 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39635856-549c-4cab-aa55-233542fbb83b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:20.052015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39635856-549c-4cab-aa55-233542fbb83b None admin] PATH_INFO: `/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/b11abed8d4424a1e9957c3dc9aefadd5/roles/de7f9ded8f5f43ff9ba38752e2139702` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:20.056239 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39635856-549c-4cab-aa55-233542fbb83b None admin] RBAC: Authorizing `identity:create_grant(project_id=fce6ee19f58b4a468c9a5d169e29a018, user_id=b11abed8d4424a1e9957c3dc9aefadd5, role_id=de7f9ded8f5f43ff9ba38752e2139702)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:20.059100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39635856-549c-4cab-aa55-233542fbb83b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:20.083505 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 490/960] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:43:19 2022] PUT /identity/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/b11abed8d4424a1e9957c3dc9aefadd5/roles/de7f9ded8f5f43ff9ba38752e2139702 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:43:21.943674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73d483-a7ec-4c57-9028-94882ed89c33 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:21.944255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73d483-a7ec-4c57-9028-94882ed89c33 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:21.944423 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73d483-a7ec-4c57-9028-94882ed89c33 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:21.946897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 471/961] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:21.952784 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d3fbd-380f-42b6-b3c8-085765c315d6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:21.953272 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d3fbd-380f-42b6-b3c8-085765c315d6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:21.953351 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d3fbd-380f-42b6-b3c8-085765c315d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:21.973334 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e79d3fbd-380f-42b6-b3c8-085765c315d6 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:21.974522 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e79d3fbd-380f-42b6-b3c8-085765c315d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:22.009096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:22.052790 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 491/962] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:22.184615 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59ec9a-0b97-4518-94b3-199817a12372 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:22.185026 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59ec9a-0b97-4518-94b3-199817a12372 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:22.185202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59ec9a-0b97-4518-94b3-199817a12372 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:22.186833 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 472/963] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:22.193167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f9176-387e-4940-940e-3308b9a4e1e5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:22.193770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f9176-387e-4940-940e-3308b9a4e1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:22.194928 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f9176-387e-4940-940e-3308b9a4e1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:22.215476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-bd3f9176-387e-4940-940e-3308b9a4e1e5 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:22.217251 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bd3f9176-387e-4940-940e-3308b9a4e1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:22.233307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31475b9-f7bc-41b9-bab8-7168a4db9dc9 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:22.233626 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31475b9-f7bc-41b9-bab8-7168a4db9dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:22.233860 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31475b9-f7bc-41b9-bab8-7168a4db9dc9 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:22.235622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 473/964] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:22.244131 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51868968-cb6f-4975-8d5d-4663a3ed2301 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:22.244131 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51868968-cb6f-4975-8d5d-4663a3ed2301 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:22.244131 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51868968-cb6f-4975-8d5d-4663a3ed2301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:22.267763 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-51868968-cb6f-4975-8d5d-4663a3ed2301 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:22.269052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-51868968-cb6f-4975-8d5d-4663a3ed2301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:22.302732 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 492/965] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:22.354314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f9176-387e-4940-940e-3308b9a4e1e5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:22.356593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 474/966] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:43:22.484133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:22.484133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f9176-387e-4940-940e-3308b9a4e1e5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:22.572692 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e7ded-b2d0-4eb9-ab92-7d205775732e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:22.573249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e7ded-b2d0-4eb9-ab92-7d205775732e None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:22.573530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e7ded-b2d0-4eb9-ab92-7d205775732e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:22.573855 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e7ded-b2d0-4eb9-ab92-7d205775732e None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:22.574517 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e7ded-b2d0-4eb9-ab92-7d205775732e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:22.575985 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e7ded-b2d0-4eb9-ab92-7d205775732e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:22.578687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: WARNING keystone.server.flask.common [None req-174e7ded-b2d0-4eb9-ab92-7d205775732e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 08:43:22.614409 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.sql.core [None req-174e7ded-b2d0-4eb9-ab92-7d205775732e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 08:43:22.614409 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 08:43:22.614409 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [parameters: {'user_id': '5616b65d2b7548f7bd32e053fb1d0142', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 08:43:22.614409 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81475) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 08:43:22.620270 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518c757-8f63-4467-a0c9-5dbcdf0061dd None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:22.620559 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518c757-8f63-4467-a0c9-5dbcdf0061dd None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:22.620835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518c757-8f63-4467-a0c9-5dbcdf0061dd None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:22.622484 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 475/967] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:43:22.626879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-174e7ded-b2d0-4eb9-ab92-7d205775732e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 08:43:22.626879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:22.626879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 08:43:22.626879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 08:43:22.626879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 08:43:22.626879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 08:43:22.626879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 08:43:22.626879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 08:43:22.626879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 08:43:22.626879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application conn.query(q) Aug 24 08:43:22.626879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 08:43:22.626879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 08:43:22.626879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 08:43:22.626879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result.read() Aug 24 08:43:22.626879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 08:43:22.627813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 08:43:22.627813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 08:43:22.627813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 08:43:22.627813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 08:43:22.627813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 08:43:22.627813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 08:43:22.627813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 08:43:22.627813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 08:43:22.627813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:22.627813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 08:43:22.627813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:22.627813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:22.627813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 08:43:22.627813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 08:43:22.627813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 08:43:22.627813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 08:43:22.627813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 08:43:22.627813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application next(self.gen) Aug 24 08:43:22.629152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 08:43:22.629152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application yield resource Aug 24 08:43:22.629152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 08:43:22.629152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application next(self.gen) Aug 24 08:43:22.629152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 08:43:22.629152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 08:43:22.629152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 08:43:22.629152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 08:43:22.629152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 08:43:22.629152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise self.value Aug 24 08:43:22.629152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 08:43:22.629152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 08:43:22.629152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 08:43:22.629152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application session.commit() Aug 24 08:43:22.629152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 08:43:22.629152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 08:43:22.629152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 08:43:22.630062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 08:43:22.630062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 08:43:22.630062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.session.flush() Aug 24 08:43:22.630062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 08:43:22.630062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 08:43:22.630062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 08:43:22.630062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 08:43:22.630062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 08:43:22.630062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application compat.raise_( Aug 24 08:43:22.630062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 08:43:22.630062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception Aug 24 08:43:22.630062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 08:43:22.630062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 08:43:22.630062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 08:43:22.630062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 08:43:22.630062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 08:43:22.630062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 08:43:22.630952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 08:43:22.630952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 08:43:22.630952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 08:43:22.630952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 08:43:22.630952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 08:43:22.630952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 08:43:22.630952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 08:43:22.630952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 08:43:22.630952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 08:43:22.630952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 08:43:22.630952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 08:43:22.630952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 08:43:22.630952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 08:43:22.630952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 08:43:22.630952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 08:43:22.630952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception Aug 24 08:43:22.631823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 08:43:22.631823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 08:43:22.631823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 08:43:22.631823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 08:43:22.631823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 08:43:22.631823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 08:43:22.631823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 08:43:22.631823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application conn.query(q) Aug 24 08:43:22.631823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 08:43:22.631823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 08:43:22.631823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 08:43:22.631823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result.read() Aug 24 08:43:22.631823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 08:43:22.631823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 08:43:22.631823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 08:43:22.631823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 08:43:22.633429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 08:43:22.633429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 08:43:22.633429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 08:43:22.633429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 08:43:22.633429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 08:43:22.633429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 08:43:22.633429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [parameters: {'user_id': '5616b65d2b7548f7bd32e053fb1d0142', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 08:43:22.633429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 08:43:22.633429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:22.633429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:22.633429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:22.633429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:22.633429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:22.633429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:22.633429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:22.634371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:22.634371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:22.634371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:22.634371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:22.634371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:22.634371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:22.634371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:22.634371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 08:43:22.634371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 08:43:22.634371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:22.634371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:22.634371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:43:22.634371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:22.634371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:43:22.634371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:22.634371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 08:43:22.634371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 08:43:22.635324 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 08:43:22.635324 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 08:43:22.635324 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 08:43:22.635324 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 08:43:22.635324 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 08:43:22.635324 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:22.635684 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cda3e-1b26-4082-a78b-4821128be7c6 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:22.635684 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cda3e-1b26-4082-a78b-4821128be7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:22.635684 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214cda3e-1b26-4082-a78b-4821128be7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:22.635877 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 493/968] 10.222.0.67 () {60 vars in 1131 bytes} [Wed Aug 24 08:43:22 2022] POST /identity/v3/users => generated 160 bytes in 275 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:43:22.635877 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e7ded-b2d0-4eb9-ab92-7d205775732e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:22.666167 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-214cda3e-1b26-4082-a78b-4821128be7c6 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:22.667382 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-214cda3e-1b26-4082-a78b-4821128be7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:22.768145 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 476/969] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:22.768145 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214cda3e-1b26-4082-a78b-4821128be7c6 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:22.773714 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e7ded-b2d0-4eb9-ab92-7d205775732e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:22.817342 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:22.832810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff21a69-823a-406c-92d8-c5a793580698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:22.833798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff21a69-823a-406c-92d8-c5a793580698 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:22.834399 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff21a69-823a-406c-92d8-c5a793580698 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:22.834824 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff21a69-823a-406c-92d8-c5a793580698 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:22.844001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff21a69-823a-406c-92d8-c5a793580698 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:22.846152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff21a69-823a-406c-92d8-c5a793580698 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:22.860361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-4ff21a69-823a-406c-92d8-c5a793580698 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:43:22.860361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:22.860361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:22.860361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:22.860361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:22.860361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:22.860361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:22.860361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:22.860361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:22.860361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:22.860361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:22.860361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:22.860361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:43:22.860361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:43:22.860361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:43:22.860361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:43:22.861554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:22.861554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:22.861554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:43:22.861554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:22.861554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:43:22.861554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:22.861554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:22.861554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:22.861554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:22.861554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:22.861554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:22.861554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:22.861554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:22.861554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:22.861554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:22.861554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:22.861554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:22.861554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:22.862496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:22.862496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:22.862496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:43:22.862496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:43:22.862496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:43:22.862496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:43:22.862496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:43:22.862496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:43:22.862496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 08:43:22.862496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:22.870145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 494/970] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:43:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:22.871483 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff21a69-823a-406c-92d8-c5a793580698 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:22.905441 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:22.914956 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214cda3e-1b26-4082-a78b-4821128be7c6 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:22.964693 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10593db6-82a9-4d1b-9516-fd7dc9eafc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:22.965382 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10593db6-82a9-4d1b-9516-fd7dc9eafc30 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:22.965637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10593db6-82a9-4d1b-9516-fd7dc9eafc30 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:22.965821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10593db6-82a9-4d1b-9516-fd7dc9eafc30 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:22.966473 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10593db6-82a9-4d1b-9516-fd7dc9eafc30 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:22.967841 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10593db6-82a9-4d1b-9516-fd7dc9eafc30 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:22.986826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 477/971] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 24 08:43:22 2022] POST /identity/v3/services => generated 242 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:43:22.987675 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10593db6-82a9-4d1b-9516-fd7dc9eafc30 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:23.030489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff21a69-823a-406c-92d8-c5a793580698 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:23.090347 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190d11f-81a8-4f15-b5d9-ea78eefe4248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:23.090903 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190d11f-81a8-4f15-b5d9-ea78eefe4248 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:23.091184 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190d11f-81a8-4f15-b5d9-ea78eefe4248 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:23.091449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190d11f-81a8-4f15-b5d9-ea78eefe4248 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:23.092059 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190d11f-81a8-4f15-b5d9-ea78eefe4248 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:23.093554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190d11f-81a8-4f15-b5d9-ea78eefe4248 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:23.105852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-b190d11f-81a8-4f15-b5d9-ea78eefe4248 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:43:23.105852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:23.105852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:43:23.105852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:43:23.105852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:43:23.105852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:43:23.105852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:43:23.105852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:23.105852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:43:23.105852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:23.105852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:23.105852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:23.105852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:23.105852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:23.105852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:23.105852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:23.105852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:23.107289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:23.107289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:23.107289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:23.107289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:23.107289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:23.107289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:23.107289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:43:23.107289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:43:23.107289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:43:23.107289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:43:23.107289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:23.107289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:23.107289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:23.107289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:23.107289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:23.107289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:23.108376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:23.108376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:23.108376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:23.108376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:23.108376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:23.108376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:23.108376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:23.108376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:23.108376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:23.108376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:23.108376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:43:23.108376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:23.108376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:43:23.108376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:23.115823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 495/972] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 24 08:43:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:23.115823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190d11f-81a8-4f15-b5d9-ea78eefe4248 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:23.132925 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10593db6-82a9-4d1b-9516-fd7dc9eafc30 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:23.143157 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:23.186867 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e220a-221f-436f-bcf9-837fe28fa6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:23.187508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e220a-221f-436f-bcf9-837fe28fa6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:23.187789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e220a-221f-436f-bcf9-837fe28fa6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:23.188064 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e220a-221f-436f-bcf9-837fe28fa6d9 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:23.188814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e220a-221f-436f-bcf9-837fe28fa6d9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:23.189229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e220a-221f-436f-bcf9-837fe28fa6d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:23.191254 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e220a-221f-436f-bcf9-837fe28fa6d9 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:23.236166 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 478/973] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 24 08:43:22 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:23.236166 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e220a-221f-436f-bcf9-837fe28fa6d9 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:23.238666 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:23.266541 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190d11f-81a8-4f15-b5d9-ea78eefe4248 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:23.307673 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45e4aa-9a04-43b1-a3f1-a4c92ffdc502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:23.308659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45e4aa-9a04-43b1-a3f1-a4c92ffdc502 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:23.309095 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45e4aa-9a04-43b1-a3f1-a4c92ffdc502 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:23.309523 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45e4aa-9a04-43b1-a3f1-a4c92ffdc502 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:23.334125 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45e4aa-9a04-43b1-a3f1-a4c92ffdc502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:43:23.341051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-cc45e4aa-9a04-43b1-a3f1-a4c92ffdc502 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:43:23.341051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:23.341051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:23.341051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:23.341051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:23.341051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:23.341051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:23.341051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:23.341051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:23.341051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:23.341051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:23.341051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:23.341051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:43:23.341051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:43:23.341051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:43:23.341051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:43:23.342069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:43:23.342069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:43:23.342069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:43:23.342069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:43:23.342069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:23.342069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:23.342069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:23.342069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:23.342069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:23.342069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:23.342069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:23.342069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:23.342069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:23.342069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:23.342069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:23.342069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:23.342933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:23.342933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:23.342933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:23.342933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:23.342933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:43:23.342933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:43:23.342933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:43:23.342933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:43:23.342933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:43:23.342933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:43:23.342933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:43:23.342933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:23.345831 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 496/974] 10.222.0.67 () {56 vars in 1120 bytes} [Wed Aug 24 08:43:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:23.352154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45e4aa-9a04-43b1-a3f1-a4c92ffdc502 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:23.393542 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e220a-221f-436f-bcf9-837fe28fa6d9 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:23.442930 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978dfb89-5e49-432c-9b6c-79f475a5b46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:23.446008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978dfb89-5e49-432c-9b6c-79f475a5b46d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:23.446479 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978dfb89-5e49-432c-9b6c-79f475a5b46d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:23.446834 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978dfb89-5e49-432c-9b6c-79f475a5b46d None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:23.447526 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978dfb89-5e49-432c-9b6c-79f475a5b46d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:23.447833 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978dfb89-5e49-432c-9b6c-79f475a5b46d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:23.449498 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978dfb89-5e49-432c-9b6c-79f475a5b46d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:23.465734 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 479/975] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 24 08:43:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:23.471544 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978dfb89-5e49-432c-9b6c-79f475a5b46d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:23.485407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45e4aa-9a04-43b1-a3f1-a4c92ffdc502 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:23.510917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:23.535746 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79b11d-969b-43ed-bffc-24ee7c4b3f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:23.537497 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79b11d-969b-43ed-bffc-24ee7c4b3f29 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:23.537497 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79b11d-969b-43ed-bffc-24ee7c4b3f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:23.537497 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79b11d-969b-43ed-bffc-24ee7c4b3f29 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:23.537773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79b11d-969b-43ed-bffc-24ee7c4b3f29 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:23.538004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79b11d-969b-43ed-bffc-24ee7c4b3f29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:23.540005 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79b11d-969b-43ed-bffc-24ee7c4b3f29 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:23.558062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 497/976] 10.222.0.67 () {56 vars in 1115 bytes} [Wed Aug 24 08:43:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:23.566441 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79b11d-969b-43ed-bffc-24ee7c4b3f29 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:23.570621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:23.663866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978dfb89-5e49-432c-9b6c-79f475a5b46d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:23.714319 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79b11d-969b-43ed-bffc-24ee7c4b3f29 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:23.725315 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711a6bb-c837-44c6-b531-9d1f29e8842d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:23.725992 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711a6bb-c837-44c6-b531-9d1f29e8842d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:23.726387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711a6bb-c837-44c6-b531-9d1f29e8842d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:23.726872 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711a6bb-c837-44c6-b531-9d1f29e8842d None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:23.727526 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0711a6bb-c837-44c6-b531-9d1f29e8842d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:23.727825 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0711a6bb-c837-44c6-b531-9d1f29e8842d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:23.729247 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0711a6bb-c837-44c6-b531-9d1f29e8842d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:23.740145 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 480/977] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 24 08:43:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:23.741131 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711a6bb-c837-44c6-b531-9d1f29e8842d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:23.784416 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192e86a-189b-4497-8c22-64dc840ad57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:23.785256 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192e86a-189b-4497-8c22-64dc840ad57e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:23.785713 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192e86a-189b-4497-8c22-64dc840ad57e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:23.786106 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192e86a-189b-4497-8c22-64dc840ad57e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:23.805681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192e86a-189b-4497-8c22-64dc840ad57e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:23.807181 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192e86a-189b-4497-8c22-64dc840ad57e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:23.824209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-0192e86a-189b-4497-8c22-64dc840ad57e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:23.824209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:23.824209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:23.824209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:23.824209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:23.824209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:23.824209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:23.824209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:23.824209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:23.824209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:23.824209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:23.824209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:23.824209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:43:23.824209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:43:23.824209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:43:23.824209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:43:23.825816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:23.825816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:23.825816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:23.825816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:23.825816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:23.825816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:23.825816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:23.825816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:23.825816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:23.825816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:23.825816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:23.825816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:23.825816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:23.825816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:23.825816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:23.825816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:23.825816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:43:23.826713 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:43:23.826713 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:23.826713 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:23.826713 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 498/978] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:23.833396 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192e86a-189b-4497-8c22-64dc840ad57e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:23.896753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:23.906840 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:23.911997 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711a6bb-c837-44c6-b531-9d1f29e8842d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:23.965736 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192e86a-189b-4497-8c22-64dc840ad57e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:23.972340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b6455-eca7-4607-aeb2-1176bf22374c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:23.973318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b6455-eca7-4607-aeb2-1176bf22374c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:23.973835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b6455-eca7-4607-aeb2-1176bf22374c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:23.974221 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b6455-eca7-4607-aeb2-1176bf22374c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:23.975400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b6455-eca7-4607-aeb2-1176bf22374c None admin] RBAC: Adding query filter params (interface=admin, service_id=1b514de88e5c414ea53ba7f49864d482, region_id=RegionOne) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:23.976527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b6455-eca7-4607-aeb2-1176bf22374c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:23.979041 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b6455-eca7-4607-aeb2-1176bf22374c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:23.988205 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 481/979] 10.222.0.67 () {56 vars in 1249 bytes} [Wed Aug 24 08:43:23 2022] GET /identity/v3/endpoints?service_id=1b514de88e5c414ea53ba7f49864d482&interface=admin®ion_id=RegionOne => generated 193 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:23.996171 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b6455-eca7-4607-aeb2-1176bf22374c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:24.023155 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08823f-5dee-45ba-889b-1d320d689f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:24.024106 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08823f-5dee-45ba-889b-1d320d689f63 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:24.024588 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08823f-5dee-45ba-889b-1d320d689f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:24.025037 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08823f-5dee-45ba-889b-1d320d689f63 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:24.026217 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08823f-5dee-45ba-889b-1d320d689f63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:24.026731 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08823f-5dee-45ba-889b-1d320d689f63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:24.029104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08823f-5dee-45ba-889b-1d320d689f63 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:24.056137 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 499/980] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:24.061917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08823f-5dee-45ba-889b-1d320d689f63 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:24.173938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b6455-eca7-4607-aeb2-1176bf22374c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:24.236065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08823f-5dee-45ba-889b-1d320d689f63 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:24.238845 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af984797-2529-41a7-a6c1-9c58d63f87f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:24.239872 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af984797-2529-41a7-a6c1-9c58d63f87f0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:24.240284 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af984797-2529-41a7-a6c1-9c58d63f87f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:24.240585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af984797-2529-41a7-a6c1-9c58d63f87f0 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:24.241630 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af984797-2529-41a7-a6c1-9c58d63f87f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:24.243501 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af984797-2529-41a7-a6c1-9c58d63f87f0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:24.250642 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 482/981] 10.222.0.67 () {56 vars in 1087 bytes} [Wed Aug 24 08:43:23 2022] GET /identity/v3/services => generated 1727 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:43:24.259898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:24.298611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60100f3e-4c60-48c0-a25b-b2bdef2c5ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:24.299831 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60100f3e-4c60-48c0-a25b-b2bdef2c5ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:24.300469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60100f3e-4c60-48c0-a25b-b2bdef2c5ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:24.301049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60100f3e-4c60-48c0-a25b-b2bdef2c5ac5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:24.313221 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60100f3e-4c60-48c0-a25b-b2bdef2c5ac5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:24.315521 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60100f3e-4c60-48c0-a25b-b2bdef2c5ac5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:24.330266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-60100f3e-4c60-48c0-a25b-b2bdef2c5ac5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:43:24.330266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:24.330266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:24.330266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:24.330266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:24.330266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:24.330266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:24.330266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:24.330266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:24.330266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:24.330266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:24.330266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:24.330266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:43:24.330266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:43:24.330266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:43:24.330266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:43:24.331389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:24.331389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:24.331389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:43:24.331389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:24.331389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:43:24.331389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:24.331389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:24.331389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:24.331389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:24.331389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:24.331389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:24.331389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:24.331389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:24.331389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:24.331389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:24.331389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:24.331389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:24.331389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:24.337645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:24.337645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:24.337645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:43:24.337645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:43:24.337645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:43:24.337645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:43:24.337645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:43:24.337645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:43:24.337645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 08:43:24.337645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:24.337645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 500/982] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 24 08:43:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:43:24.341013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af984797-2529-41a7-a6c1-9c58d63f87f0 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:24.365593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:24.467543 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af984797-2529-41a7-a6c1-9c58d63f87f0 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:24.504087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778852f7-6aeb-4038-ad22-4ea16ebf632a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:24.504623 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778852f7-6aeb-4038-ad22-4ea16ebf632a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:24.504903 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778852f7-6aeb-4038-ad22-4ea16ebf632a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:24.505182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778852f7-6aeb-4038-ad22-4ea16ebf632a None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:24.505915 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778852f7-6aeb-4038-ad22-4ea16ebf632a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:24.506227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778852f7-6aeb-4038-ad22-4ea16ebf632a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:24.508245 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778852f7-6aeb-4038-ad22-4ea16ebf632a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:24.528063 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 483/983] 10.222.0.67 () {56 vars in 1149 bytes} [Wed Aug 24 08:43:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:24.534200 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60100f3e-4c60-48c0-a25b-b2bdef2c5ac5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:24.652966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60100f3e-4c60-48c0-a25b-b2bdef2c5ac5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:24.694049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585542cb-9339-4f3a-b280-f6599982fb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:24.694946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585542cb-9339-4f3a-b280-f6599982fb7a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:24.695265 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585542cb-9339-4f3a-b280-f6599982fb7a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:24.695540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585542cb-9339-4f3a-b280-f6599982fb7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:24.755131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:24.785358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585542cb-9339-4f3a-b280-f6599982fb7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:24.786822 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585542cb-9339-4f3a-b280-f6599982fb7a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:24.903761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 501/984] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:43:24 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:43:24.908151 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778852f7-6aeb-4038-ad22-4ea16ebf632a None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:24.997363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:25.025333 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778852f7-6aeb-4038-ad22-4ea16ebf632a None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:25.060510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf3683-670d-4da4-8f0b-f80731f6a445 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:25.061477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf3683-670d-4da4-8f0b-f80731f6a445 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:25.061984 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf3683-670d-4da4-8f0b-f80731f6a445 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:25.063765 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 502/985] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:43:25.068754 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9afb1d0-7a52-46af-af19-07bf3cc5f950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:25.070057 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb1021-de1a-4409-ad2c-76f5162e513d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:25.070335 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afb1d0-7a52-46af-af19-07bf3cc5f950 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:25.070482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb1021-de1a-4409-ad2c-76f5162e513d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:25.070604 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb1021-de1a-4409-ad2c-76f5162e513d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:25.070747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afb1d0-7a52-46af-af19-07bf3cc5f950 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:25.071304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afb1d0-7a52-46af-af19-07bf3cc5f950 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:25.088793 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9afb1d0-7a52-46af-af19-07bf3cc5f950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:25.090278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9afb1d0-7a52-46af-af19-07bf3cc5f950 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:25.093981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-88bb1021-de1a-4409-ad2c-76f5162e513d None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:25.096706 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-88bb1021-de1a-4409-ad2c-76f5162e513d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:25.107385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-c9afb1d0-7a52-46af-af19-07bf3cc5f950 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:25.107385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:25.107385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:25.107385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:25.107385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:25.107385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:25.107385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:25.107385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:25.107385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:25.107385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:25.107385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:25.107385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:25.107385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:43:25.107385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:43:25.107385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:43:25.107385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:43:25.108836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:25.108836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:25.108836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:25.108836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:25.108836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:25.108836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:25.108836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:25.108836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:25.108836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:25.108836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:25.108836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:25.108836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:25.108836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:25.108836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:25.108836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:25.108836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:25.108836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:43:25.109837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:43:25.109837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:25.109837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:25.109837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 484/986] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:25.114686 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9afb1d0-7a52-46af-af19-07bf3cc5f950 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:25.196067 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 503/987] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:25 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:43:25.283374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9afb1d0-7a52-46af-af19-07bf3cc5f950 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:25.336339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:25.348255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446a9d3-bd04-4a40-9302-f12127cf5c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:25.349273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446a9d3-bd04-4a40-9302-f12127cf5c14 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:25.349752 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446a9d3-bd04-4a40-9302-f12127cf5c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:25.350228 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446a9d3-bd04-4a40-9302-f12127cf5c14 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:25.352217 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446a9d3-bd04-4a40-9302-f12127cf5c14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:25.352706 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446a9d3-bd04-4a40-9302-f12127cf5c14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:25.355358 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446a9d3-bd04-4a40-9302-f12127cf5c14 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:25.384142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 485/988] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:25.387655 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb1021-de1a-4409-ad2c-76f5162e513d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:25.393530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:25.466205 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446a9d3-bd04-4a40-9302-f12127cf5c14 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:25.572725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb1021-de1a-4409-ad2c-76f5162e513d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:25.579813 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446a9d3-bd04-4a40-9302-f12127cf5c14 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:25.619053 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38a354-8442-4a27-bbef-ac289dbc5ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:25.619696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38a354-8442-4a27-bbef-ac289dbc5ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:25.619968 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38a354-8442-4a27-bbef-ac289dbc5ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:25.620273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38a354-8442-4a27-bbef-ac289dbc5ec6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:25.620863 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38a354-8442-4a27-bbef-ac289dbc5ec6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:25.622231 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38a354-8442-4a27-bbef-ac289dbc5ec6 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:25.631349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-de38a354-8442-4a27-bbef-ac289dbc5ec6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:43:25.631349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:25.631349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:43:25.631349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:43:25.631349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:43:25.631349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:43:25.631349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:43:25.631349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:25.631349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:43:25.631349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:25.631349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:25.631349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:25.631349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:25.631349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:25.631349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:25.631349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:25.631349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:25.634717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:25.634717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:25.634717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:25.634717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:25.634717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:25.634717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:25.634717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:43:25.634717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:43:25.634717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:43:25.634717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:43:25.634717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:25.634717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:25.634717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:25.634717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:25.634717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:25.634717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:25.636808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:25.636808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:25.636808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:25.636808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:25.636808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:25.636808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:25.636808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:25.636808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:25.636808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:25.636808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:25.636808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:43:25.636808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:25.636808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:43:25.636808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:25.636808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 486/989] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:43:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:43:25.639593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3f988-7bf7-48e4-8d5c-f4c32e121d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:25.639593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3f988-7bf7-48e4-8d5c-f4c32e121d0e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:25.639593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3f988-7bf7-48e4-8d5c-f4c32e121d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:25.639593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3f988-7bf7-48e4-8d5c-f4c32e121d0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:25.640543 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38a354-8442-4a27-bbef-ac289dbc5ec6 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:25.647169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3f988-7bf7-48e4-8d5c-f4c32e121d0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:25.650325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3f988-7bf7-48e4-8d5c-f4c32e121d0e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:25.664916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-0ab3f988-7bf7-48e4-8d5c-f4c32e121d0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:25.664916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:25.664916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:25.664916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:25.664916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:25.664916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:25.664916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:25.664916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:25.664916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:25.664916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:25.664916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:25.664916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:25.664916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:43:25.664916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:43:25.664916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:43:25.666563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:43:25.666563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:25.666563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:25.666563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:25.666563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:25.666563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:25.666563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:25.666563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:25.666563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:25.666563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:25.666563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:25.666563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:25.666563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:25.666563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:25.666563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:25.666563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:25.666563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:25.671646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:43:25.671646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:43:25.671646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:43:25.671646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:43:25.671646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:43:25.671646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:43:25.671646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:25.671646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:25.671646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 504/990] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:43:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:25.674321 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3f988-7bf7-48e4-8d5c-f4c32e121d0e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:25.708968 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:25.765010 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:25.772246 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38a354-8442-4a27-bbef-ac289dbc5ec6 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:25.823446 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3f988-7bf7-48e4-8d5c-f4c32e121d0e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:25.827020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd33336-032e-4502-b3f2-14e49e0a540e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:25.828172 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd33336-032e-4502-b3f2-14e49e0a540e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:25.828751 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd33336-032e-4502-b3f2-14e49e0a540e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:25.829308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd33336-032e-4502-b3f2-14e49e0a540e None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:25.830556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd33336-032e-4502-b3f2-14e49e0a540e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:25.831157 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd33336-032e-4502-b3f2-14e49e0a540e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:25.833933 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd33336-032e-4502-b3f2-14e49e0a540e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:25.851182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 487/991] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 24 08:43:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:25.853148 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd33336-032e-4502-b3f2-14e49e0a540e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:25.885693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738866f6-c425-4c65-9bd3-e745c3c16e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:25.886609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738866f6-c425-4c65-9bd3-e745c3c16e8b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:25.887027 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738866f6-c425-4c65-9bd3-e745c3c16e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:25.887426 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738866f6-c425-4c65-9bd3-e745c3c16e8b None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:25.888619 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738866f6-c425-4c65-9bd3-e745c3c16e8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:25.889056 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738866f6-c425-4c65-9bd3-e745c3c16e8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:25.890823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738866f6-c425-4c65-9bd3-e745c3c16e8b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:25.906932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 505/992] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:43:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:25.912803 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738866f6-c425-4c65-9bd3-e745c3c16e8b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:26.015391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd33336-032e-4502-b3f2-14e49e0a540e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:26.036539 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:26.059261 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05577b-de89-4b91-9f0c-bacb489e8fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:26.059879 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05577b-de89-4b91-9f0c-bacb489e8fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:26.060192 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05577b-de89-4b91-9f0c-bacb489e8fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:26.060468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05577b-de89-4b91-9f0c-bacb489e8fc5 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:26.060919 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738866f6-c425-4c65-9bd3-e745c3c16e8b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:26.061609 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05577b-de89-4b91-9f0c-bacb489e8fc5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:26.061609 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05577b-de89-4b91-9f0c-bacb489e8fc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:26.062742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05577b-de89-4b91-9f0c-bacb489e8fc5 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:26.068384 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 488/993] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 24 08:43:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:26.074358 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05577b-de89-4b91-9f0c-bacb489e8fc5 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:26.100411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:26.129606 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008b02c-85b9-4bb3-a63f-035df7d05265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:26.131254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008b02c-85b9-4bb3-a63f-035df7d05265 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:26.131727 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008b02c-85b9-4bb3-a63f-035df7d05265 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:26.132949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008b02c-85b9-4bb3-a63f-035df7d05265 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:26.135011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008b02c-85b9-4bb3-a63f-035df7d05265 None admin] RBAC: Adding query filter params (role.id=de7f9ded8f5f43ff9ba38752e2139702, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=b11abed8d4424a1e9957c3dc9aefadd5) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:26.135536 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008b02c-85b9-4bb3-a63f-035df7d05265 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:26.139521 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008b02c-85b9-4bb3-a63f-035df7d05265 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:26.164126 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 506/994] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:43:25 2022] GET /identity/v3/role_assignments?user.id=b11abed8d4424a1e9957c3dc9aefadd5&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=de7f9ded8f5f43ff9ba38752e2139702 => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:26.194657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05577b-de89-4b91-9f0c-bacb489e8fc5 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:26.237316 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ce46a-9e25-4faa-bd23-69035eec7e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:26.237959 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ce46a-9e25-4faa-bd23-69035eec7e76 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:26.238287 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ce46a-9e25-4faa-bd23-69035eec7e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:26.238562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ce46a-9e25-4faa-bd23-69035eec7e76 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:26.239281 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ce46a-9e25-4faa-bd23-69035eec7e76 None admin] RBAC: Adding query filter params (interface=public, service_id=9e4c1f07faf04ababeb9397157577af0, region_id=RegionOne) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:26.239588 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ce46a-9e25-4faa-bd23-69035eec7e76 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:26.241015 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ce46a-9e25-4faa-bd23-69035eec7e76 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:26.252652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 489/995] 10.222.0.67 () {56 vars in 1251 bytes} [Wed Aug 24 08:43:26 2022] GET /identity/v3/endpoints?service_id=9e4c1f07faf04ababeb9397157577af0&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:26.253462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008b02c-85b9-4bb3-a63f-035df7d05265 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:26.317274 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4549c1-bce5-4b41-8a0b-aeaa3a52560b None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:26.317634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4549c1-bce5-4b41-8a0b-aeaa3a52560b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:26.317832 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4549c1-bce5-4b41-8a0b-aeaa3a52560b None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:26.319213 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 490/996] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:43:26.326114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c9404-87d9-4091-9a39-73a1ee888531 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:26.326417 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c9404-87d9-4091-9a39-73a1ee888531 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:26.326754 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c9404-87d9-4091-9a39-73a1ee888531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:26.347821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-0d2c9404-87d9-4091-9a39-73a1ee888531 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:26.349079 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0d2c9404-87d9-4091-9a39-73a1ee888531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:26.386296 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008b02c-85b9-4bb3-a63f-035df7d05265 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:26.420460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:26.457562 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787a9de-f690-4788-8e20-7d56d2589a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:26.458621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787a9de-f690-4788-8e20-7d56d2589a95 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:26.459133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787a9de-f690-4788-8e20-7d56d2589a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:26.460609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787a9de-f690-4788-8e20-7d56d2589a95 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:26.460748 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 491/997] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:43:26.461353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787a9de-f690-4788-8e20-7d56d2589a95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:26.463889 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787a9de-f690-4788-8e20-7d56d2589a95 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:26.478198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 507/998] 10.222.0.67 () {56 vars in 1087 bytes} [Wed Aug 24 08:43:26 2022] GET /identity/v3/services => generated 1727 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:43:26.735828 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c9404-87d9-4091-9a39-73a1ee888531 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:26.871239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c9404-87d9-4091-9a39-73a1ee888531 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:26.923829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658455f-679f-42f8-a9ca-f2a214096c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:26.924976 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658455f-679f-42f8-a9ca-f2a214096c4b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:26.925475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658455f-679f-42f8-a9ca-f2a214096c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:26.925954 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658455f-679f-42f8-a9ca-f2a214096c4b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:26.926884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658455f-679f-42f8-a9ca-f2a214096c4b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:26.929378 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658455f-679f-42f8-a9ca-f2a214096c4b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:26.940568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-5658455f-679f-42f8-a9ca-f2a214096c4b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:43:26.940568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:26.940568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:43:26.940568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:43:26.940568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:43:26.940568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:43:26.940568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:43:26.940568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:26.940568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:43:26.940568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:26.940568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:26.940568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:26.940568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:26.940568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:26.940568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:26.940568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:26.940568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:26.942581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:26.942581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:26.942581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:26.942581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:26.942581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:26.942581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:26.942581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:43:26.942581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:43:26.942581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:43:26.942581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:43:26.942581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:26.942581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:26.942581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:26.942581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:26.942581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:26.942581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:26.944395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:26.944395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:26.944395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:26.944395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:26.944395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:26.944395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:26.944395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:26.944395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:26.944395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:26.944395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:26.944395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:43:26.944395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:26.944395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 08:43:26.944395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:26.944395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 492/999] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 24 08:43:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:26.953262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787a9de-f690-4788-8e20-7d56d2589a95 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:27.027882 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:27.117613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787a9de-f690-4788-8e20-7d56d2589a95 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:27.183749 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cccf55-d016-44b0-86c6-aa2d23e84ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:27.184809 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cccf55-d016-44b0-86c6-aa2d23e84ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:27.185304 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cccf55-d016-44b0-86c6-aa2d23e84ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:27.185769 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cccf55-d016-44b0-86c6-aa2d23e84ee0 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:27.186892 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cccf55-d016-44b0-86c6-aa2d23e84ee0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:27.187411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cccf55-d016-44b0-86c6-aa2d23e84ee0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:27.190117 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cccf55-d016-44b0-86c6-aa2d23e84ee0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:27.205866 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 508/1000] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 24 08:43:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:27.207751 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658455f-679f-42f8-a9ca-f2a214096c4b None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:27.273965 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:27.349875 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658455f-679f-42f8-a9ca-f2a214096c4b None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:27.402726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f7f07-de5e-4e00-bb22-a236fb2f891d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:27.403382 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f7f07-de5e-4e00-bb22-a236fb2f891d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:27.403655 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f7f07-de5e-4e00-bb22-a236fb2f891d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:27.403930 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f7f07-de5e-4e00-bb22-a236fb2f891d None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:27.404660 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f7f07-de5e-4e00-bb22-a236fb2f891d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:27.404963 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f7f07-de5e-4e00-bb22-a236fb2f891d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:27.406500 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f7f07-de5e-4e00-bb22-a236fb2f891d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:27.413256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 493/1001] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 24 08:43:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:27.419745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cccf55-d016-44b0-86c6-aa2d23e84ee0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:27.612189 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cccf55-d016-44b0-86c6-aa2d23e84ee0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:27.642791 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:27.671015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcccc0ab-ca1c-491f-acb1-bd70331e8afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:27.671657 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccc0ab-ca1c-491f-acb1-bd70331e8afd None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:27.671947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccc0ab-ca1c-491f-acb1-bd70331e8afd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:27.672254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccc0ab-ca1c-491f-acb1-bd70331e8afd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:27.673201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcccc0ab-ca1c-491f-acb1-bd70331e8afd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:27.674802 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcccc0ab-ca1c-491f-acb1-bd70331e8afd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:27.704158 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 509/1002] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:43:27 2022] POST /identity/v3/endpoints => generated 325 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:43:28.277749 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea4aa0-254a-43b6-9d8f-3174237ccfbc None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:28.278555 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea4aa0-254a-43b6-9d8f-3174237ccfbc None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:28.278728 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea4aa0-254a-43b6-9d8f-3174237ccfbc None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:28.280368 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 494/1003] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:28.286345 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387c581-17eb-4cf3-8585-76be1ad2edbb None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:28.287078 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387c581-17eb-4cf3-8585-76be1ad2edbb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:28.287367 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387c581-17eb-4cf3-8585-76be1ad2edbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:28.308813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f387c581-17eb-4cf3-8585-76be1ad2edbb None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:28.310196 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f387c581-17eb-4cf3-8585-76be1ad2edbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:28.403581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 510/1004] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:28 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:28.626783 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f88aa2-e61e-4ef9-b931-5cff8961ccd1 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:28.627020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f88aa2-e61e-4ef9-b931-5cff8961ccd1 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:28.627399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f88aa2-e61e-4ef9-b931-5cff8961ccd1 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:28.628866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 495/1005] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:43:28.634692 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26053a-6cd0-4632-9db6-19a18106d3ef None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:28.634963 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26053a-6cd0-4632-9db6-19a18106d3ef None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:28.635280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26053a-6cd0-4632-9db6-19a18106d3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:28.655761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-fb26053a-6cd0-4632-9db6-19a18106d3ef None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:28.657336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fb26053a-6cd0-4632-9db6-19a18106d3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:28.684481 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f88aa2-e61e-4ef9-b931-5cff8961ccd1 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:28.755834 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 511/1006] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:28 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:28.814948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f88aa2-e61e-4ef9-b931-5cff8961ccd1 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:28.818068 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:28.858226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086293f5-9a3f-464b-a604-191b93dc4e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:28.858881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086293f5-9a3f-464b-a604-191b93dc4e09 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:28.859159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086293f5-9a3f-464b-a604-191b93dc4e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:28.860948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086293f5-9a3f-464b-a604-191b93dc4e09 None admin] PATH_INFO: `/v3/services/image` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:28.860948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086293f5-9a3f-464b-a604-191b93dc4e09 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:28.861614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086293f5-9a3f-464b-a604-191b93dc4e09 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:28.871616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-086293f5-9a3f-464b-a604-191b93dc4e09 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:43:28.871616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:28.871616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:43:28.871616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:43:28.871616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:43:28.871616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:43:28.871616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:43:28.871616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:28.871616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:43:28.871616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:28.871616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:28.871616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:28.871616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:28.871616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:28.871616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:28.871616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:28.871616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:28.876556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:28.876556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:28.876556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:28.876556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:28.876556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:28.876556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:28.876556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:43:28.876556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:43:28.876556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:43:28.876556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:43:28.876556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:28.876556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:28.876556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:28.876556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:28.876556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:28.876556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:28.877562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:28.877562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:28.877562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:28.877562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:28.877562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:28.877562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:28.877562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:28.877562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:28.877562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:28.877562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:28.877562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:43:28.877562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:28.877562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:43:28.877562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:28.877562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 496/1007] 10.222.0.67 () {56 vars in 1105 bytes} [Wed Aug 24 08:43:28 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:43:28.880211 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26053a-6cd0-4632-9db6-19a18106d3ef None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:28.917859 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:29.037715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26053a-6cd0-4632-9db6-19a18106d3ef None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:29.098874 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4adf59-1282-4c6f-8b07-a26604531b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:29.099897 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4adf59-1282-4c6f-8b07-a26604531b0a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:29.100397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4adf59-1282-4c6f-8b07-a26604531b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:29.101398 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4adf59-1282-4c6f-8b07-a26604531b0a None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:29.102827 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4adf59-1282-4c6f-8b07-a26604531b0a None admin] RBAC: Adding query filter params (name=image) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:29.103339 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4adf59-1282-4c6f-8b07-a26604531b0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:29.105992 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4adf59-1282-4c6f-8b07-a26604531b0a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:29.120263 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 512/1008] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:29.122184 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086293f5-9a3f-464b-a604-191b93dc4e09 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:29.230327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4adf59-1282-4c6f-8b07-a26604531b0a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:29.246642 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086293f5-9a3f-464b-a604-191b93dc4e09 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:29.286243 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f51687-b359-4689-b303-3088eadec8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:29.287782 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f51687-b359-4689-b303-3088eadec8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:29.288416 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f51687-b359-4689-b303-3088eadec8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:29.288915 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f51687-b359-4689-b303-3088eadec8d7 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:29.289953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f51687-b359-4689-b303-3088eadec8d7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:29.290546 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f51687-b359-4689-b303-3088eadec8d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:29.292352 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f51687-b359-4689-b303-3088eadec8d7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:29.297879 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 497/1009] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:29 2022] GET /identity/v3/services?type=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:29.417321 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:29.431956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4adf59-1282-4c6f-8b07-a26604531b0a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:29.498947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78550de3-6cda-41f1-8675-9023927d1c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:29.499950 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78550de3-6cda-41f1-8675-9023927d1c2d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:29.500482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78550de3-6cda-41f1-8675-9023927d1c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:29.500967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78550de3-6cda-41f1-8675-9023927d1c2d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:29.502046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78550de3-6cda-41f1-8675-9023927d1c2d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:29.504685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78550de3-6cda-41f1-8675-9023927d1c2d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:29.524489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-78550de3-6cda-41f1-8675-9023927d1c2d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:43:29.524489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:29.524489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:43:29.524489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:43:29.524489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:43:29.524489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:43:29.524489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:43:29.524489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:29.524489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:43:29.524489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:29.524489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:29.524489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:29.524489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:29.524489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:29.524489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:29.524489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:29.524489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:29.525621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:29.525621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:29.525621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:29.525621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:29.525621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:29.525621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:29.525621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:43:29.525621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:43:29.525621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:43:29.525621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:43:29.525621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:29.525621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:29.525621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:29.525621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:29.525621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:29.525621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:29.526496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:29.526496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:29.526496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:29.526496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:29.526496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:29.526496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:29.526496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:29.526496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:29.526496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:29.526496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:29.526496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:43:29.526496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:29.526496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 08:43:29.526496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:29.527239 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 513/1010] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:43:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:43:29.536683 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f51687-b359-4689-b303-3088eadec8d7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:29.577129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:29.659607 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f51687-b359-4689-b303-3088eadec8d7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:29.704129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a78cad-2cb9-4142-80cd-b405a8ae466e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:29.705360 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a78cad-2cb9-4142-80cd-b405a8ae466e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:29.706042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a78cad-2cb9-4142-80cd-b405a8ae466e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:29.706748 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a78cad-2cb9-4142-80cd-b405a8ae466e None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:29.707817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a78cad-2cb9-4142-80cd-b405a8ae466e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:29.708638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a78cad-2cb9-4142-80cd-b405a8ae466e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:29.711741 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a78cad-2cb9-4142-80cd-b405a8ae466e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:29.725364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 498/1011] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 24 08:43:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:29.726800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78550de3-6cda-41f1-8675-9023927d1c2d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:29.894638 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78550de3-6cda-41f1-8675-9023927d1c2d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:29.968977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0860a7dc-2127-4b4b-9fff-489e277144bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:29.970035 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860a7dc-2127-4b4b-9fff-489e277144bc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:29.970519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860a7dc-2127-4b4b-9fff-489e277144bc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:29.971004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860a7dc-2127-4b4b-9fff-489e277144bc None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:29.972143 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860a7dc-2127-4b4b-9fff-489e277144bc None admin] RBAC: Adding query filter params (type=compute) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:29.972685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860a7dc-2127-4b4b-9fff-489e277144bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:29.975025 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860a7dc-2127-4b4b-9fff-489e277144bc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:29.982767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 514/1012] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 24 08:43:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:29.988560 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a78cad-2cb9-4142-80cd-b405a8ae466e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:30.103501 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:30.111687 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a78cad-2cb9-4142-80cd-b405a8ae466e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:30.158362 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea083c-f1e5-426b-9a89-74f919a856ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:30.158882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea083c-f1e5-426b-9a89-74f919a856ba None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:30.159173 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea083c-f1e5-426b-9a89-74f919a856ba None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:30.159427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea083c-f1e5-426b-9a89-74f919a856ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:30.160150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea083c-f1e5-426b-9a89-74f919a856ba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:30.161550 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea083c-f1e5-426b-9a89-74f919a856ba None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:30.196156 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 499/1013] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:43:29 2022] POST /identity/v3/endpoints => generated 334 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:43:30.387495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77ca63-619f-4d74-85ef-e1a3c4a3ae0a None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:30.387838 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77ca63-619f-4d74-85ef-e1a3c4a3ae0a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:30.388185 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77ca63-619f-4d74-85ef-e1a3c4a3ae0a None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:30.390002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 515/1014] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:30.396568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a98eaa-f881-4f40-a2d7-03e9ab5d1227 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:30.397093 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a98eaa-f881-4f40-a2d7-03e9ab5d1227 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:30.397181 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a98eaa-f881-4f40-a2d7-03e9ab5d1227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:30.421498 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-92a98eaa-f881-4f40-a2d7-03e9ab5d1227 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:30.422784 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-92a98eaa-f881-4f40-a2d7-03e9ab5d1227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:30.526299 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 500/1015] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:30 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:30.917680 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77ca63-619f-4d74-85ef-e1a3c4a3ae0a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:30.972842 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:31.087868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77ca63-619f-4d74-85ef-e1a3c4a3ae0a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:31.126212 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3c4ba-48e8-4f63-bd2e-9a4bf8eb2cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:31.127062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3c4ba-48e8-4f63-bd2e-9a4bf8eb2cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:31.127417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3c4ba-48e8-4f63-bd2e-9a4bf8eb2cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:31.127723 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3c4ba-48e8-4f63-bd2e-9a4bf8eb2cd2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:31.129156 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c3c4ba-48e8-4f63-bd2e-9a4bf8eb2cd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:31.130868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c3c4ba-48e8-4f63-bd2e-9a4bf8eb2cd2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:31.134056 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 516/1016] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:31.140518 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a98eaa-f881-4f40-a2d7-03e9ab5d1227 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:31.247320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:31.276720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a98eaa-f881-4f40-a2d7-03e9ab5d1227 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:31.315717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ad202-8dc5-4c32-bcd6-7e4a45ede679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:31.316365 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ad202-8dc5-4c32-bcd6-7e4a45ede679 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:31.316652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ad202-8dc5-4c32-bcd6-7e4a45ede679 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:31.316914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ad202-8dc5-4c32-bcd6-7e4a45ede679 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:31.317580 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ad202-8dc5-4c32-bcd6-7e4a45ede679 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:31.319030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ad202-8dc5-4c32-bcd6-7e4a45ede679 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:31.364231 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 501/1017] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 24 08:43:31 2022] POST /identity/v3/projects => generated 311 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:43:31.482798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8db1a-0118-444e-922e-0e77eb29e4a8 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:31.483089 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8db1a-0118-444e-922e-0e77eb29e4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:31.483383 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8db1a-0118-444e-922e-0e77eb29e4a8 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:31.485247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 517/1018] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:31.492164 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbdc390-760d-4c70-8ae1-2dde7ff0513e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:31.492732 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbdc390-760d-4c70-8ae1-2dde7ff0513e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:31.492732 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbdc390-760d-4c70-8ae1-2dde7ff0513e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:31.521050 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-cdbdc390-760d-4c70-8ae1-2dde7ff0513e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:31.522345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-cdbdc390-760d-4c70-8ae1-2dde7ff0513e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:31.624440 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 502/1019] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:31 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:43:31.895826 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8db1a-0118-444e-922e-0e77eb29e4a8 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:32.009272 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:32.022508 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8db1a-0118-444e-922e-0e77eb29e4a8 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:32.066071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367bf25-c796-469f-9c1c-0c036f3d7b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:32.066685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367bf25-c796-469f-9c1c-0c036f3d7b3a None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:32.066965 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367bf25-c796-469f-9c1c-0c036f3d7b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:32.067328 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367bf25-c796-469f-9c1c-0c036f3d7b3a None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:32.067906 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367bf25-c796-469f-9c1c-0c036f3d7b3a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:32.070150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367bf25-c796-469f-9c1c-0c036f3d7b3a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:32.091937 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 518/1020] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 24 08:43:31 2022] POST /identity/v3/services => generated 242 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:43:32.376987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43531bc-ca9f-428e-a6d8-8a49cedb6c62 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:32.377362 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43531bc-ca9f-428e-a6d8-8a49cedb6c62 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:32.377516 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43531bc-ca9f-428e-a6d8-8a49cedb6c62 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:32.379137 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 503/1021] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:43:32.388388 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84634bdb-05c2-49b1-87dd-3d2c74047b1a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:32.388880 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84634bdb-05c2-49b1-87dd-3d2c74047b1a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:32.389353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84634bdb-05c2-49b1-87dd-3d2c74047b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:32.419076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-84634bdb-05c2-49b1-87dd-3d2c74047b1a None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:32.421113 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-84634bdb-05c2-49b1-87dd-3d2c74047b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:32.556166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 519/1022] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:32.938477 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43531bc-ca9f-428e-a6d8-8a49cedb6c62 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:33.016227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:33.072612 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43531bc-ca9f-428e-a6d8-8a49cedb6c62 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:33.113411 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15bb75-03c7-46ce-a635-81658d8351fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:33.114066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15bb75-03c7-46ce-a635-81658d8351fe None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:33.114348 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15bb75-03c7-46ce-a635-81658d8351fe None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:33.114635 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15bb75-03c7-46ce-a635-81658d8351fe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:33.128694 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15bb75-03c7-46ce-a635-81658d8351fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:43:33.134743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-5d15bb75-03c7-46ce-a635-81658d8351fe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:43:33.134743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:33.134743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:33.134743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:33.134743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:33.134743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:33.134743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:33.134743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:33.134743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:33.134743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:33.134743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:33.134743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:33.134743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:43:33.134743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:43:33.134743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:43:33.134743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:43:33.136435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:43:33.136435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:43:33.136435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:43:33.136435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:43:33.136435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:33.136435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:33.136435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:33.136435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:33.136435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:33.136435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:33.136435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:33.136435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:33.136435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:33.136435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:33.136435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:33.136435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:33.137887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:33.137887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:33.137887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:33.137887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:33.137887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:43:33.137887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:43:33.137887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:43:33.137887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:43:33.137887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:43:33.137887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:43:33.137887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:43:33.137887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:33.137887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 504/1023] 10.222.0.67 () {56 vars in 1120 bytes} [Wed Aug 24 08:43:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:33.144155 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84634bdb-05c2-49b1-87dd-3d2c74047b1a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:33.237347 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:33.266757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84634bdb-05c2-49b1-87dd-3d2c74047b1a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:33.305879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc7ab5-f03c-437d-a9d4-30e5588d1c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:33.306501 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc7ab5-f03c-437d-a9d4-30e5588d1c5d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:33.306793 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc7ab5-f03c-437d-a9d4-30e5588d1c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:33.307066 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc7ab5-f03c-437d-a9d4-30e5588d1c5d None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:33.307728 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc7ab5-f03c-437d-a9d4-30e5588d1c5d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:33.308049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc7ab5-f03c-437d-a9d4-30e5588d1c5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:33.309428 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc7ab5-f03c-437d-a9d4-30e5588d1c5d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:33.318587 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 520/1024] 10.222.0.67 () {56 vars in 1115 bytes} [Wed Aug 24 08:43:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:33.325000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15bb75-03c7-46ce-a635-81658d8351fe None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:33.463355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15bb75-03c7-46ce-a635-81658d8351fe None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:33.486442 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:33.504749 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816d699-69ba-4c9e-a3fc-48c5abe31263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:33.505401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816d699-69ba-4c9e-a3fc-48c5abe31263 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:33.505687 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816d699-69ba-4c9e-a3fc-48c5abe31263 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:33.505955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816d699-69ba-4c9e-a3fc-48c5abe31263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:33.506102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d0670-e8cb-4c81-9441-25fb5d64645d None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:33.506402 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d0670-e8cb-4c81-9441-25fb5d64645d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:33.506675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d0670-e8cb-4c81-9441-25fb5d64645d None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:33.508932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 521/1025] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:33.514588 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45895f95-3e22-4a63-b9a3-d96a71f5e86b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:33.514909 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45895f95-3e22-4a63-b9a3-d96a71f5e86b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:33.515138 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45895f95-3e22-4a63-b9a3-d96a71f5e86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:33.520434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816d699-69ba-4c9e-a3fc-48c5abe31263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:33.522186 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816d699-69ba-4c9e-a3fc-48c5abe31263 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:33.536388 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-45895f95-3e22-4a63-b9a3-d96a71f5e86b None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:33.537784 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-45895f95-3e22-4a63-b9a3-d96a71f5e86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:33.544853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-b816d699-69ba-4c9e-a3fc-48c5abe31263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:33.544853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:33.544853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:33.544853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:33.544853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:33.544853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:33.544853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:33.544853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:33.544853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:33.544853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:33.544853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:33.544853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:33.544853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:43:33.544853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:43:33.544853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:43:33.544853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:43:33.547130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:33.547130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:33.547130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:33.547130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:33.547130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:33.547130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:33.547130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:33.547130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:33.547130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:33.547130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:33.547130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:33.547130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:33.547130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:33.547130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:33.547130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:33.547130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:33.547130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:43:33.551282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:43:33.551282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:33.551282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:33.551282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 505/1026] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:33.552609 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816d699-69ba-4c9e-a3fc-48c5abe31263 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:33.640456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 522/1027] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:33.676814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816d699-69ba-4c9e-a3fc-48c5abe31263 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:33.728417 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f9fd0-4235-4590-9872-2665475cae01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:33.729648 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f9fd0-4235-4590-9872-2665475cae01 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:33.730245 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f9fd0-4235-4590-9872-2665475cae01 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:33.730777 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f9fd0-4235-4590-9872-2665475cae01 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:33.732217 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f9fd0-4235-4590-9872-2665475cae01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:33.732660 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f9fd0-4235-4590-9872-2665475cae01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:33.735686 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f9fd0-4235-4590-9872-2665475cae01 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:33.764806 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 506/1028] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:33.766342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45895f95-3e22-4a63-b9a3-d96a71f5e86b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:33.878781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:33.919330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f9fd0-4235-4590-9872-2665475cae01 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:33.938923 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45895f95-3e22-4a63-b9a3-d96a71f5e86b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:33.965119 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:33.998370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea25cc-4d61-409c-a0b5-887f131f3aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:33.999221 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea25cc-4d61-409c-a0b5-887f131f3aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:34.000227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea25cc-4d61-409c-a0b5-887f131f3aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:34.000446 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea25cc-4d61-409c-a0b5-887f131f3aa3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:34.013933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea25cc-4d61-409c-a0b5-887f131f3aa3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:34.016326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea25cc-4d61-409c-a0b5-887f131f3aa3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:34.035365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-82ea25cc-4d61-409c-a0b5-887f131f3aa3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:43:34.035365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:34.035365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:34.035365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:34.035365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:34.035365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:34.035365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:34.035365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:34.035365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:34.035365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:34.035365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:34.035365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:34.035365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:43:34.035365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:43:34.035365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:43:34.035365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:43:34.036920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:34.036920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:34.036920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:43:34.036920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:34.036920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:43:34.036920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:34.036920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:34.036920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:34.036920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:34.036920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:34.036920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:34.036920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:34.036920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:34.036920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:34.036920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:34.036920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:34.036920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:34.036920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:34.039397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:34.039397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:34.039397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:43:34.039397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:43:34.039397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:43:34.039397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:43:34.039397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:43:34.039397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:43:34.039397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:43:34.039397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:34.043333 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 523/1029] 10.222.0.67 () {56 vars in 1093 bytes} [Wed Aug 24 08:43:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:34.044973 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea25cc-4d61-409c-a0b5-887f131f3aa3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:34.073867 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f9fd0-4235-4590-9872-2665475cae01 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:34.119959 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd00ec-abf6-439c-9903-63260a03658d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:34.120939 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd00ec-abf6-439c-9903-63260a03658d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:34.121368 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd00ec-abf6-439c-9903-63260a03658d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:34.121656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd00ec-abf6-439c-9903-63260a03658d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:34.122851 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd00ec-abf6-439c-9903-63260a03658d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:34.124364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd00ec-abf6-439c-9903-63260a03658d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:34.131191 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 507/1030] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:34.132694 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd00ec-abf6-439c-9903-63260a03658d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:34.198432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea25cc-4d61-409c-a0b5-887f131f3aa3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:34.233017 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:34.241290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc08751-732a-4c0e-8952-793a3e78d4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:34.242094 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08751-732a-4c0e-8952-793a3e78d4cb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:34.242474 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08751-732a-4c0e-8952-793a3e78d4cb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:34.242774 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08751-732a-4c0e-8952-793a3e78d4cb None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:34.243654 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc08751-732a-4c0e-8952-793a3e78d4cb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:34.243967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc08751-732a-4c0e-8952-793a3e78d4cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:34.245667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc08751-732a-4c0e-8952-793a3e78d4cb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:34.270032 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 524/1031] 10.222.0.67 () {56 vars in 1133 bytes} [Wed Aug 24 08:43:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:34.279424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc08751-732a-4c0e-8952-793a3e78d4cb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:34.291463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:34.295058 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd00ec-abf6-439c-9903-63260a03658d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:34.336583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332f056-07b0-4eff-a445-9ca4147c85b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:34.337224 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332f056-07b0-4eff-a445-9ca4147c85b7 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:34.337509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332f056-07b0-4eff-a445-9ca4147c85b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:34.337782 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332f056-07b0-4eff-a445-9ca4147c85b7 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:34.338451 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332f056-07b0-4eff-a445-9ca4147c85b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:34.339920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332f056-07b0-4eff-a445-9ca4147c85b7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:34.377153 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 508/1032] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 24 08:43:34 2022] POST /identity/v3/users => generated 285 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:43:34.377153 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce8a25-e61c-4834-8837-290bd106b18e None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:34.377153 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce8a25-e61c-4834-8837-290bd106b18e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:34.377153 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce8a25-e61c-4834-8837-290bd106b18e None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:34.377153 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 509/1033] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:34.388229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3eaea-932f-4d4f-8987-d7d07c008416 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:34.388229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3eaea-932f-4d4f-8987-d7d07c008416 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:34.388229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3eaea-932f-4d4f-8987-d7d07c008416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:34.417579 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc08751-732a-4c0e-8952-793a3e78d4cb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:34.421980 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f0d3eaea-932f-4d4f-8987-d7d07c008416 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:34.424292 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f0d3eaea-932f-4d4f-8987-d7d07c008416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:34.463124 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfc175-6dfd-4cfb-88db-b592fbe164b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:34.463979 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfc175-6dfd-4cfb-88db-b592fbe164b0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:34.464555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfc175-6dfd-4cfb-88db-b592fbe164b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:34.464773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfc175-6dfd-4cfb-88db-b592fbe164b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:34.533656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 510/1034] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:34.560644 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:34.583086 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dfc175-6dfd-4cfb-88db-b592fbe164b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:34.585810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dfc175-6dfd-4cfb-88db-b592fbe164b0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:34.758480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 525/1035] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:43:34 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 08:43:34.760044 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3eaea-932f-4d4f-8987-d7d07c008416 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:34.845902 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfc175-6dfd-4cfb-88db-b592fbe164b0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:34.866158 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:34.885279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3eaea-932f-4d4f-8987-d7d07c008416 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:34.925515 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81d39c-561b-4e8a-941a-52eaf8fb8276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:34.926269 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81d39c-561b-4e8a-941a-52eaf8fb8276 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:34.926597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81d39c-561b-4e8a-941a-52eaf8fb8276 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:34.926918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81d39c-561b-4e8a-941a-52eaf8fb8276 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:34.940977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81d39c-561b-4e8a-941a-52eaf8fb8276 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:34.942627 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81d39c-561b-4e8a-941a-52eaf8fb8276 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:34.961111 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:34.961426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-8a81d39c-561b-4e8a-941a-52eaf8fb8276 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:34.961426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:34.961426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:34.961426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:34.961426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:34.961426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:34.961426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:34.961426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:34.961426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:34.961426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:34.961426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:34.961426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:34.961426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:43:34.961426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:43:34.961426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:43:34.961426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:43:34.962328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:34.962328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:34.962328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:34.962328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:34.962328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:34.962328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:34.962328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:34.962328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:34.962328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:34.962328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:34.962328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:34.962328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:34.962328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:34.962328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:34.962328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:34.962328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:34.962328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:43:34.963243 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:43:34.963243 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:34.963243 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:34.963243 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 511/1036] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:34.969948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81d39c-561b-4e8a-941a-52eaf8fb8276 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:34.978866 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfc175-6dfd-4cfb-88db-b592fbe164b0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:35.018668 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab0564-ddf1-4032-b0e5-df509b30ba7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:35.019209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab0564-ddf1-4032-b0e5-df509b30ba7e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:35.019496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab0564-ddf1-4032-b0e5-df509b30ba7e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:35.019770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab0564-ddf1-4032-b0e5-df509b30ba7e None admin] PATH_INFO: `/v3/services/image` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:35.020403 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab0564-ddf1-4032-b0e5-df509b30ba7e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:35.021783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab0564-ddf1-4032-b0e5-df509b30ba7e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:35.034947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-59ab0564-ddf1-4032-b0e5-df509b30ba7e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:43:35.034947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:35.034947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:43:35.034947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:43:35.034947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:43:35.034947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:43:35.034947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:43:35.034947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:35.034947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:43:35.034947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:35.034947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:35.034947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:35.034947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:35.034947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:35.034947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:35.034947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:35.034947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:35.036242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:35.036242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:35.036242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:35.036242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:35.036242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:35.036242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:35.036242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:43:35.036242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:43:35.036242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:43:35.036242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:43:35.036242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:35.036242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:35.036242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:35.036242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:35.036242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:35.036242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:35.037337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:35.037337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:35.037337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:35.037337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:35.037337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:35.037337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:35.037337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:35.037337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:35.037337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:35.037337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:35.037337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:43:35.037337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:35.037337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:43:35.037337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:35.046086 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 526/1037] 10.222.0.67 () {56 vars in 1105 bytes} [Wed Aug 24 08:43:34 2022] GET /identity/v3/services/image => generated 86 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:43:35.047933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab0564-ddf1-4032-b0e5-df509b30ba7e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:35.114130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81d39c-561b-4e8a-941a-52eaf8fb8276 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:35.158775 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058314f3-bc24-4ec6-a408-a34e839404c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:35.159391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058314f3-bc24-4ec6-a408-a34e839404c0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:35.159674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058314f3-bc24-4ec6-a408-a34e839404c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:35.159989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058314f3-bc24-4ec6-a408-a34e839404c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:35.160675 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058314f3-bc24-4ec6-a408-a34e839404c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:35.160981 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058314f3-bc24-4ec6-a408-a34e839404c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:35.162425 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058314f3-bc24-4ec6-a408-a34e839404c0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:35.171700 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:35.183942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 512/1038] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:43:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:35.186571 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058314f3-bc24-4ec6-a408-a34e839404c0 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:35.208555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab0564-ddf1-4032-b0e5-df509b30ba7e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:35.269564 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695992d9-9d4b-4bdf-b40b-c93835c92583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:35.270773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695992d9-9d4b-4bdf-b40b-c93835c92583 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:35.271350 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695992d9-9d4b-4bdf-b40b-c93835c92583 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:35.271841 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695992d9-9d4b-4bdf-b40b-c93835c92583 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:35.273101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695992d9-9d4b-4bdf-b40b-c93835c92583 None admin] RBAC: Adding query filter params (name=image) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:35.273656 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695992d9-9d4b-4bdf-b40b-c93835c92583 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:35.276338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695992d9-9d4b-4bdf-b40b-c93835c92583 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:35.290916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 527/1039] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:35.293587 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695992d9-9d4b-4bdf-b40b-c93835c92583 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:35.323166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:35.325909 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058314f3-bc24-4ec6-a408-a34e839404c0 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:35.389361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcacd0-8df5-4c51-b6c4-d799cbc34cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:35.390233 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcacd0-8df5-4c51-b6c4-d799cbc34cef None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:35.390575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcacd0-8df5-4c51-b6c4-d799cbc34cef None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:35.390958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcacd0-8df5-4c51-b6c4-d799cbc34cef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:35.402858 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dcacd0-8df5-4c51-b6c4-d799cbc34cef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:35.405307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dcacd0-8df5-4c51-b6c4-d799cbc34cef None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:35.427305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-69dcacd0-8df5-4c51-b6c4-d799cbc34cef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:35.427305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:35.427305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:35.427305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:35.427305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:35.427305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:35.427305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:35.427305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:35.427305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:35.427305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:35.427305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:35.427305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:35.427305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:43:35.427305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:43:35.427305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:43:35.429460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:43:35.429460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:35.429460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:35.429460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:35.429460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:35.429460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:35.429460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:35.429460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:35.429460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:35.429460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:35.429460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:35.429460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:35.429460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:35.429460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:35.429460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:35.429460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:35.429460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:35.430400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:43:35.430400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:43:35.430400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:43:35.430400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:43:35.430400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:43:35.430400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:43:35.430400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:35.430400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:35.430400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 513/1040] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:43:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:35.436335 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcacd0-8df5-4c51-b6c4-d799cbc34cef None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:35.453838 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695992d9-9d4b-4bdf-b40b-c93835c92583 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:35.501870 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1564ee-9db5-4f60-ba79-fbf9cc948652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:35.502689 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1564ee-9db5-4f60-ba79-fbf9cc948652 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:35.503156 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1564ee-9db5-4f60-ba79-fbf9cc948652 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:35.503567 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1564ee-9db5-4f60-ba79-fbf9cc948652 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:35.504349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1564ee-9db5-4f60-ba79-fbf9cc948652 None admin] RBAC: Adding query filter params (type=image) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:35.504846 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1564ee-9db5-4f60-ba79-fbf9cc948652 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:35.506364 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1564ee-9db5-4f60-ba79-fbf9cc948652 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:35.509889 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:35.516129 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 528/1041] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:35.521023 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1564ee-9db5-4f60-ba79-fbf9cc948652 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:35.564502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcacd0-8df5-4c51-b6c4-d799cbc34cef None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:35.611788 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e40f0-c1b4-4f7e-a914-38c009984afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:35.613029 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e40f0-c1b4-4f7e-a914-38c009984afe None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:35.613029 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e40f0-c1b4-4f7e-a914-38c009984afe None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:35.613029 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e40f0-c1b4-4f7e-a914-38c009984afe None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:35.613686 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e40f0-c1b4-4f7e-a914-38c009984afe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:35.614068 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e40f0-c1b4-4f7e-a914-38c009984afe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:35.615536 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e40f0-c1b4-4f7e-a914-38c009984afe None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:35.626024 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:35.634320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 514/1042] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:43:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:35.640559 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e40f0-c1b4-4f7e-a914-38c009984afe None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:35.648584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1564ee-9db5-4f60-ba79-fbf9cc948652 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:35.687256 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d16bb-f899-4a8e-a8a4-7c12ec12bf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:35.688316 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d16bb-f899-4a8e-a8a4-7c12ec12bf6b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:35.688605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d16bb-f899-4a8e-a8a4-7c12ec12bf6b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:35.688872 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d16bb-f899-4a8e-a8a4-7c12ec12bf6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:35.689590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d16bb-f899-4a8e-a8a4-7c12ec12bf6b None admin] RBAC: Adding query filter params (interface=public, service_id=b8b8609b1c6e4995ac15c9a8a54cceb6, region_id=RegionOne) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:35.689897 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d16bb-f899-4a8e-a8a4-7c12ec12bf6b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:35.691267 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d16bb-f899-4a8e-a8a4-7c12ec12bf6b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:35.698130 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 529/1043] 10.222.0.67 () {56 vars in 1251 bytes} [Wed Aug 24 08:43:35 2022] GET /identity/v3/endpoints?service_id=b8b8609b1c6e4995ac15c9a8a54cceb6&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:35.703717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d16bb-f899-4a8e-a8a4-7c12ec12bf6b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:35.769203 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e40f0-c1b4-4f7e-a914-38c009984afe None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:35.788301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:35.808983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c35865-a9a0-4177-9638-b654bc6ca684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:35.809602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c35865-a9a0-4177-9638-b654bc6ca684 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:35.809878 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c35865-a9a0-4177-9638-b654bc6ca684 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:35.810294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c35865-a9a0-4177-9638-b654bc6ca684 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:35.810890 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c35865-a9a0-4177-9638-b654bc6ca684 None admin] RBAC: Adding query filter params (role.id=de7f9ded8f5f43ff9ba38752e2139702, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=ca56dd7216fb4946946b2801e144fec2) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:35.811189 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c35865-a9a0-4177-9638-b654bc6ca684 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:35.812662 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c35865-a9a0-4177-9638-b654bc6ca684 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:35.828126 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 515/1044] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:43:35 2022] GET /identity/v3/role_assignments?user.id=ca56dd7216fb4946946b2801e144fec2&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=de7f9ded8f5f43ff9ba38752e2139702 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:35.852609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d16bb-f899-4a8e-a8a4-7c12ec12bf6b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:35.901365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a516a-4ef3-4000-b9fb-81ba505ffa28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:35.901987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a516a-4ef3-4000-b9fb-81ba505ffa28 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:35.902267 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a516a-4ef3-4000-b9fb-81ba505ffa28 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:35.902538 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a516a-4ef3-4000-b9fb-81ba505ffa28 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:35.903160 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a516a-4ef3-4000-b9fb-81ba505ffa28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:35.904601 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a516a-4ef3-4000-b9fb-81ba505ffa28 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:35.912864 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 530/1045] 10.222.0.67 () {56 vars in 1087 bytes} [Wed Aug 24 08:43:35 2022] GET /identity/v3/services => generated 1957 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:43:36.664309 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11125c-9890-461a-b496-adc4bc367b98 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:36.665156 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11125c-9890-461a-b496-adc4bc367b98 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:36.665484 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11125c-9890-461a-b496-adc4bc367b98 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:36.666913 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 516/1046] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:36.673721 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978755fe-8cc5-410c-881e-1616aa483ec2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:36.674282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978755fe-8cc5-410c-881e-1616aa483ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:36.674282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978755fe-8cc5-410c-881e-1616aa483ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:36.697626 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-978755fe-8cc5-410c-881e-1616aa483ec2 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:36.698834 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-978755fe-8cc5-410c-881e-1616aa483ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:36.701051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:36.807476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 531/1047] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:43:37.153865 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11125c-9890-461a-b496-adc4bc367b98 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:37.284057 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11125c-9890-461a-b496-adc4bc367b98 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:37.330014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c3466-63b6-4e6d-a2d1-c9239b7c934c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:37.330680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c3466-63b6-4e6d-a2d1-c9239b7c934c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:37.330957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c3466-63b6-4e6d-a2d1-c9239b7c934c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:37.331257 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c3466-63b6-4e6d-a2d1-c9239b7c934c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:37.344927 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c3466-63b6-4e6d-a2d1-c9239b7c934c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:43:37.349674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-bd3c3466-63b6-4e6d-a2d1-c9239b7c934c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:43:37.349674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:37.349674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:37.349674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:37.349674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:37.349674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:37.349674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:37.349674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:37.349674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:37.349674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:37.349674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:37.349674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:37.349674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:43:37.349674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:43:37.349674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:43:37.349674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:43:37.350719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:43:37.350719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:43:37.350719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:43:37.350719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:43:37.350719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:37.350719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:37.350719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:37.350719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:37.350719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:37.350719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:37.350719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:37.350719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:37.350719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:37.350719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:37.350719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:37.350719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:37.352758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:37.352758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:37.352758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:37.352758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:37.352758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:43:37.352758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:43:37.352758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:43:37.352758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:43:37.352758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:43:37.352758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:43:37.352758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:43:37.352758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:37.360068 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 517/1048] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:43:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:37.360379 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978755fe-8cc5-410c-881e-1616aa483ec2 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:37.479479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978755fe-8cc5-410c-881e-1616aa483ec2 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:37.519706 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7aa907-00d3-4faa-a567-ef35d9b47c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:37.520362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7aa907-00d3-4faa-a567-ef35d9b47c61 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:37.520642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7aa907-00d3-4faa-a567-ef35d9b47c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:37.520912 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7aa907-00d3-4faa-a567-ef35d9b47c61 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:37.521598 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7aa907-00d3-4faa-a567-ef35d9b47c61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:37.521900 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7aa907-00d3-4faa-a567-ef35d9b47c61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:37.523867 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7aa907-00d3-4faa-a567-ef35d9b47c61 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:37.530068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:37.540588 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 532/1049] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:43:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:37.545154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c3466-63b6-4e6d-a2d1-c9239b7c934c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:37.608294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:37.677021 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c3466-63b6-4e6d-a2d1-c9239b7c934c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:37.721175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bd678-314e-44d0-b7dd-46f8b36d6b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:37.722040 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bd678-314e-44d0-b7dd-46f8b36d6b0d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:37.722319 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bd678-314e-44d0-b7dd-46f8b36d6b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:37.722590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bd678-314e-44d0-b7dd-46f8b36d6b0d None admin] PATH_INFO: `/v3/users/6719b6bc9b4c4afe862a2f1ffe359a88` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:37.740967 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786bd678-314e-44d0-b7dd-46f8b36d6b0d None admin] RBAC: Authorizing `identity:get_user(user_id=6719b6bc9b4c4afe862a2f1ffe359a88)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:37.743720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786bd678-314e-44d0-b7dd-46f8b36d6b0d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:37.745935 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 518/1050] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:43:37 2022] GET /identity/v3/users/6719b6bc9b4c4afe862a2f1ffe359a88 => generated 285 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:37.753294 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7aa907-00d3-4faa-a567-ef35d9b47c61 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:37.873811 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7aa907-00d3-4faa-a567-ef35d9b47c61 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:37.910808 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bcc726-e0ec-491d-a5aa-b07ab9623b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:37.911699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcc726-e0ec-491d-a5aa-b07ab9623b8c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:37.912176 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcc726-e0ec-491d-a5aa-b07ab9623b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:37.912596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcc726-e0ec-491d-a5aa-b07ab9623b8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:37.931667 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30f8f0-9352-4a88-ad37-e4aadc5ebde9 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:37.931950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30f8f0-9352-4a88-ad37-e4aadc5ebde9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:37.932256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30f8f0-9352-4a88-ad37-e4aadc5ebde9 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:37.933897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 519/1051] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:43:37.939687 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af8d59-42e7-43b2-8d72-7b91b29da716 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:37.939973 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af8d59-42e7-43b2-8d72-7b91b29da716 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:37.940349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af8d59-42e7-43b2-8d72-7b91b29da716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:37.960346 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-a9af8d59-42e7-43b2-8d72-7b91b29da716 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:37.961957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a9af8d59-42e7-43b2-8d72-7b91b29da716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:37.993665 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:37.998339 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bcc726-e0ec-491d-a5aa-b07ab9623b8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:37.999822 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bcc726-e0ec-491d-a5aa-b07ab9623b8c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:38.065696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 520/1052] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:43:38.140342 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb975d5-ee2a-4e6a-9c69-1721c68a848c None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:38.140626 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb975d5-ee2a-4e6a-9c69-1721c68a848c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:38.140898 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb975d5-ee2a-4e6a-9c69-1721c68a848c None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:38.142389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 521/1053] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:43:38.151435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 533/1054] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:43:37 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:43:38.151632 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db3093-cea1-4981-b572-6e9a13305468 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:38.151632 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db3093-cea1-4981-b572-6e9a13305468 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:38.151632 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db3093-cea1-4981-b572-6e9a13305468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:38.153227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bcc726-e0ec-491d-a5aa-b07ab9623b8c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:38.169281 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-32db3093-cea1-4981-b572-6e9a13305468 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:38.170475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-32db3093-cea1-4981-b572-6e9a13305468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:38.197364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:38.245941 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 522/1055] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:43:38.283149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:38.286932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bcc726-e0ec-491d-a5aa-b07ab9623b8c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:38.327730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb625232-89a8-4248-97eb-629a738968b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:38.328962 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb625232-89a8-4248-97eb-629a738968b5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:38.329514 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb625232-89a8-4248-97eb-629a738968b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:38.330001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb625232-89a8-4248-97eb-629a738968b5 None admin] PATH_INFO: `/v3/projects/08e8989146554b8c9c2b6c8d31ccaa10` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:38.331645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb625232-89a8-4248-97eb-629a738968b5 None admin] RBAC: Authorizing `identity:get_project(project_id=08e8989146554b8c9c2b6c8d31ccaa10)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:38.333644 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb625232-89a8-4248-97eb-629a738968b5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:38.340162 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db3093-cea1-4981-b572-6e9a13305468 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:38.343233 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 534/1056] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:43:38 2022] GET /identity/v3/projects/08e8989146554b8c9c2b6c8d31ccaa10 => generated 311 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:38.344001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb625232-89a8-4248-97eb-629a738968b5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:38.467101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db3093-cea1-4981-b572-6e9a13305468 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:38.491424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb625232-89a8-4248-97eb-629a738968b5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:38.524770 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27788d39-9d03-42db-b014-df686494abfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:38.525329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27788d39-9d03-42db-b014-df686494abfa None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:38.525607 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27788d39-9d03-42db-b014-df686494abfa None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:38.525881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27788d39-9d03-42db-b014-df686494abfa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:38.532208 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885422b4-f26f-4398-9c8a-e22c19835c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:38.532744 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885422b4-f26f-4398-9c8a-e22c19835c4b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:38.533022 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885422b4-f26f-4398-9c8a-e22c19835c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:38.533288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885422b4-f26f-4398-9c8a-e22c19835c4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:38.534119 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885422b4-f26f-4398-9c8a-e22c19835c4b None admin] RBAC: Adding query filter params (role.id=b13ffa72344c4856901b1ba003e273fd, scope.project.id=08e8989146554b8c9c2b6c8d31ccaa10, user.id=6719b6bc9b4c4afe862a2f1ffe359a88) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:38.534327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885422b4-f26f-4398-9c8a-e22c19835c4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:38.535754 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885422b4-f26f-4398-9c8a-e22c19835c4b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:38.536824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27788d39-9d03-42db-b014-df686494abfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:43:38.541535 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 535/1057] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:43:38 2022] GET /identity/v3/role_assignments?user.id=6719b6bc9b4c4afe862a2f1ffe359a88&scope.project.id=08e8989146554b8c9c2b6c8d31ccaa10&role.id=b13ffa72344c4856901b1ba003e273fd => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:38.541803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-27788d39-9d03-42db-b014-df686494abfa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:43:38.541803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:38.541803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:38.541803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:38.541803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:38.541803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:38.541803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:38.541803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:38.541803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:38.541803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:38.541803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:38.541803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:38.541803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:43:38.541803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:43:38.541803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:43:38.541803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:43:38.543098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:43:38.543098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:43:38.543098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:43:38.543098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:43:38.543098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:38.543098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:38.543098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:38.543098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:38.543098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:38.543098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:38.543098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:38.543098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:38.543098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:38.543098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:38.543098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:38.543098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:38.544074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:38.544074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:38.544074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:38.544074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:38.544074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:43:38.544074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:43:38.544074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:43:38.544074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:43:38.544074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:43:38.544074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:43:38.544074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:43:38.544074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:38.544074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 523/1058] 10.222.0.67 () {56 vars in 1120 bytes} [Wed Aug 24 08:43:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:43:38.553568 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885422b4-f26f-4398-9c8a-e22c19835c4b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:38.553904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27788d39-9d03-42db-b014-df686494abfa None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:38.583876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:38.596860 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:38.689464 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27788d39-9d03-42db-b014-df686494abfa None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:38.694880 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885422b4-f26f-4398-9c8a-e22c19835c4b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:38.746661 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51ed9d-d3bc-4830-8624-e7b6bb327ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:38.748172 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51ed9d-d3bc-4830-8624-e7b6bb327ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:38.748469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51ed9d-d3bc-4830-8624-e7b6bb327ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:38.748951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51ed9d-d3bc-4830-8624-e7b6bb327ea4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:38.749468 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51ed9d-d3bc-4830-8624-e7b6bb327ea4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:38.750971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51ed9d-d3bc-4830-8624-e7b6bb327ea4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:38.752256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdf232-c023-4c69-9caa-21e29f57f8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:38.752984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdf232-c023-4c69-9caa-21e29f57f8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:38.753167 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdf232-c023-4c69-9caa-21e29f57f8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:38.753419 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdf232-c023-4c69-9caa-21e29f57f8c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:38.754084 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdf232-c023-4c69-9caa-21e29f57f8c3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:38.754383 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdf232-c023-4c69-9caa-21e29f57f8c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:38.755773 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdf232-c023-4c69-9caa-21e29f57f8c3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:38.761629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-bb51ed9d-d3bc-4830-8624-e7b6bb327ea4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:43:38.761629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:38.761629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:43:38.761629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:43:38.761629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:43:38.761629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:43:38.761629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:43:38.761629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:38.761629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:43:38.761629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:38.761629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:43:38.761629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:38.761629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:38.761629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:38.761629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:38.761629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:38.761629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:38.763219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:38.763219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:38.763219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:38.763219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:38.763219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:38.763219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:38.763219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:43:38.763219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:43:38.763219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:43:38.763219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:43:38.763219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:38.763219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:38.763219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:38.763219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:38.763219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:38.763219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:38.764298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:38.764298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:38.764298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:38.764298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:38.764298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:38.764298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:38.764298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:38.764298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:38.764298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:38.764298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:38.764298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:43:38.764298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:43:38.764298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 08:43:38.764298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:38.765062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 536/1059] 10.222.0.67 () {56 vars in 1105 bytes} [Wed Aug 24 08:43:38 2022] GET /identity/v3/services/image => generated 86 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:38.768212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 524/1060] 10.222.0.67 () {56 vars in 1115 bytes} [Wed Aug 24 08:43:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:38.773877 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51ed9d-d3bc-4830-8624-e7b6bb327ea4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:38.774041 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdf232-c023-4c69-9caa-21e29f57f8c3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:38.871210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:38.886807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:38.892553 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdf232-c023-4c69-9caa-21e29f57f8c3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:38.894292 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51ed9d-d3bc-4830-8624-e7b6bb327ea4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:38.930145 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e11f7f-e347-45db-8c00-29ae4b0491ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:38.930789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e11f7f-e347-45db-8c00-29ae4b0491ce None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:38.931057 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e11f7f-e347-45db-8c00-29ae4b0491ce None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:38.931317 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e11f7f-e347-45db-8c00-29ae4b0491ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:38.949665 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0dc15-1903-4b78-b14c-e83288a93e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:38.949915 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e11f7f-e347-45db-8c00-29ae4b0491ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:38.950283 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0dc15-1903-4b78-b14c-e83288a93e30 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:38.950560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0dc15-1903-4b78-b14c-e83288a93e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:38.950857 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0dc15-1903-4b78-b14c-e83288a93e30 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:38.951498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f0dc15-1903-4b78-b14c-e83288a93e30 None admin] RBAC: Adding query filter params (name=image) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:38.951797 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f0dc15-1903-4b78-b14c-e83288a93e30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:38.952712 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e11f7f-e347-45db-8c00-29ae4b0491ce None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:38.953186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f0dc15-1903-4b78-b14c-e83288a93e30 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:38.958847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 537/1061] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:38.970071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0dc15-1903-4b78-b14c-e83288a93e30 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:38.985511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-c3e11f7f-e347-45db-8c00-29ae4b0491ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:38.985511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:38.985511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:38.985511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:38.985511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:38.985511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:38.985511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:38.985511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:38.985511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:38.985511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:38.985511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:38.985511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:38.985511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:43:38.985511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:43:38.985511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:43:38.985511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:43:38.986521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:38.986521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:38.986521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:38.986521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:38.986521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:38.986521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:38.986521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:38.986521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:38.986521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:38.986521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:38.986521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:38.986521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:38.986521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:38.986521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:38.986521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:38.986521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:38.986521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:43:38.987409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:43:38.987409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:38.987409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:38.988353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 525/1062] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:43:38.993196 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e11f7f-e347-45db-8c00-29ae4b0491ce None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:39.114456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0dc15-1903-4b78-b14c-e83288a93e30 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:39.129656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e11f7f-e347-45db-8c00-29ae4b0491ce None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:39.154894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca09e56-830f-4112-ba68-825d731192ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:39.154894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca09e56-830f-4112-ba68-825d731192ff None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:39.154894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca09e56-830f-4112-ba68-825d731192ff None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:39.154894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca09e56-830f-4112-ba68-825d731192ff None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:39.157652 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca09e56-830f-4112-ba68-825d731192ff None admin] RBAC: Adding query filter params (type=image) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:39.157652 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca09e56-830f-4112-ba68-825d731192ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:39.157652 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca09e56-830f-4112-ba68-825d731192ff None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:39.163195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 538/1063] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:39.169074 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca09e56-830f-4112-ba68-825d731192ff None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:39.178664 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac0b74-fea8-4000-9740-74df1354bae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:39.180326 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac0b74-fea8-4000-9740-74df1354bae7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:39.180818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac0b74-fea8-4000-9740-74df1354bae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:39.181301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac0b74-fea8-4000-9740-74df1354bae7 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:39.182514 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac0b74-fea8-4000-9740-74df1354bae7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:39.183065 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac0b74-fea8-4000-9740-74df1354bae7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:39.185655 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac0b74-fea8-4000-9740-74df1354bae7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:39.188538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:39.205783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:39.219992 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 526/1064] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:39.221225 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac0b74-fea8-4000-9740-74df1354bae7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:39.301616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca09e56-830f-4112-ba68-825d731192ff None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:39.346003 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac0b74-fea8-4000-9740-74df1354bae7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:39.350787 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd2dc0-7808-4f60-8b06-6f80dab6e31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:39.351490 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd2dc0-7808-4f60-8b06-6f80dab6e31c None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:39.351767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd2dc0-7808-4f60-8b06-6f80dab6e31c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:39.352037 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd2dc0-7808-4f60-8b06-6f80dab6e31c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:39.352760 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd2dc0-7808-4f60-8b06-6f80dab6e31c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:39.354149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd2dc0-7808-4f60-8b06-6f80dab6e31c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:39.400133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 539/1065] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:43:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:43:39.412626 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cad9b-a2e1-4824-a8aa-e9b160c851c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:39.413385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cad9b-a2e1-4824-a8aa-e9b160c851c3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:39.414579 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cad9b-a2e1-4824-a8aa-e9b160c851c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:39.414899 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cad9b-a2e1-4824-a8aa-e9b160c851c3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:39.424172 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cad9b-a2e1-4824-a8aa-e9b160c851c3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:39.425733 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cad9b-a2e1-4824-a8aa-e9b160c851c3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:39.439859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-ff3cad9b-a2e1-4824-a8aa-e9b160c851c3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:43:39.439859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:39.439859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:39.439859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:39.439859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:39.439859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:39.439859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:39.439859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:39.439859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:39.439859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:39.439859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:39.439859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:39.439859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:43:39.439859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:43:39.439859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:43:39.439859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:43:39.440918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:39.440918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:39.440918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:43:39.440918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:39.440918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:43:39.440918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:39.440918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:39.440918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:39.440918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:39.440918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:39.440918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:39.440918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:39.440918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:39.440918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:39.440918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:39.440918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:39.440918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:39.440918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:39.441874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:39.441874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:39.441874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:43:39.441874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:43:39.441874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:43:39.441874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:43:39.441874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:43:39.441874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:43:39.441874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:43:39.441874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:39.441874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 527/1066] 10.222.0.67 () {56 vars in 1093 bytes} [Wed Aug 24 08:43:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:43:39.448593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd2dc0-7808-4f60-8b06-6f80dab6e31c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:39.578776 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:39.604193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd2dc0-7808-4f60-8b06-6f80dab6e31c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:39.654433 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfa49b-aa60-4095-aca6-cd291d0f52ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:39.654991 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfa49b-aa60-4095-aca6-cd291d0f52ae None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:39.655265 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfa49b-aa60-4095-aca6-cd291d0f52ae None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:39.655591 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfa49b-aa60-4095-aca6-cd291d0f52ae None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:39.656407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfa49b-aa60-4095-aca6-cd291d0f52ae None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:39.656713 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfa49b-aa60-4095-aca6-cd291d0f52ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:39.658197 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfa49b-aa60-4095-aca6-cd291d0f52ae None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:39.678332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 540/1067] 10.222.0.67 () {56 vars in 1133 bytes} [Wed Aug 24 08:43:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:39.684444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cad9b-a2e1-4824-a8aa-e9b160c851c3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:39.752838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:39.809940 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cad9b-a2e1-4824-a8aa-e9b160c851c3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:39.852024 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50114a-b4b7-46e6-836d-79472d5b0cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:39.852699 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50114a-b4b7-46e6-836d-79472d5b0cec None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:39.852977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50114a-b4b7-46e6-836d-79472d5b0cec None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:39.853248 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50114a-b4b7-46e6-836d-79472d5b0cec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:39.874525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50114a-b4b7-46e6-836d-79472d5b0cec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:39.875996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50114a-b4b7-46e6-836d-79472d5b0cec None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:39.891881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-1a50114a-b4b7-46e6-836d-79472d5b0cec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:39.891881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:39.891881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:39.891881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:39.891881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:39.891881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:39.891881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:39.891881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:39.891881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:39.891881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:39.891881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:39.891881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:39.891881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:43:39.891881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:43:39.891881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:43:39.891881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:43:39.893176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:39.893176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:39.893176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:39.893176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:39.893176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:39.893176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:39.893176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:39.893176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:39.893176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:39.893176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:39.893176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:39.893176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:39.893176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:39.893176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:39.893176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:39.893176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:39.893176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:43:39.894106 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:43:39.894106 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:39.894106 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:39.896199 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 528/1068] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:39.899920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfa49b-aa60-4095-aca6-cd291d0f52ae None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:40.048198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfa49b-aa60-4095-aca6-cd291d0f52ae None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:40.114552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1d607-a365-4b02-bc1b-0ba1e5fd2d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:40.116057 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1d607-a365-4b02-bc1b-0ba1e5fd2d49 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:40.116057 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1d607-a365-4b02-bc1b-0ba1e5fd2d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:40.116681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1d607-a365-4b02-bc1b-0ba1e5fd2d49 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:40.117591 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1d607-a365-4b02-bc1b-0ba1e5fd2d49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:40.118009 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1d607-a365-4b02-bc1b-0ba1e5fd2d49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:40.119666 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1d607-a365-4b02-bc1b-0ba1e5fd2d49 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:40.129386 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:40.138448 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 541/1069] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:40.148452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50114a-b4b7-46e6-836d-79472d5b0cec None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:40.267452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50114a-b4b7-46e6-836d-79472d5b0cec None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:40.269194 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:40.311519 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66d583-d176-4817-9277-f7bd4524d7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:40.312065 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66d583-d176-4817-9277-f7bd4524d7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:40.312378 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66d583-d176-4817-9277-f7bd4524d7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:40.312642 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66d583-d176-4817-9277-f7bd4524d7c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:40.320536 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66d583-d176-4817-9277-f7bd4524d7c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:40.322006 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66d583-d176-4817-9277-f7bd4524d7c2 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:40.333778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-8b66d583-d176-4817-9277-f7bd4524d7c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:40.333778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:40.333778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:40.333778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:40.333778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:40.333778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:40.333778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:40.333778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:40.333778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:40.333778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:40.333778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:40.333778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:40.333778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:43:40.333778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:43:40.333778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:43:40.334882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:43:40.334882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:40.334882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:40.334882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:40.334882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:40.334882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:40.334882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:40.334882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:40.334882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:40.334882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:40.334882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:40.334882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:40.334882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:40.334882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:40.334882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:40.334882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:40.334882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:40.335952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:43:40.335952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:43:40.335952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:43:40.335952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:43:40.335952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:43:40.335952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:43:40.335952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:40.335952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:40.335952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 529/1070] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:43:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:40.342535 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1d607-a365-4b02-bc1b-0ba1e5fd2d49 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:40.469789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1d607-a365-4b02-bc1b-0ba1e5fd2d49 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:40.511582 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a778cd6-9e78-4d7d-ad5d-63a3fa3bd684 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:40.511863 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a778cd6-9e78-4d7d-ad5d-63a3fa3bd684 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:40.512156 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a778cd6-9e78-4d7d-ad5d-63a3fa3bd684 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:40.513785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 530/1071] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:43:40.517028 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a120e8-4977-4fdc-9588-1aea5f05a81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:40.518089 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a120e8-4977-4fdc-9588-1aea5f05a81f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:40.518573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a120e8-4977-4fdc-9588-1aea5f05a81f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:40.519053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a120e8-4977-4fdc-9588-1aea5f05a81f None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:40.520227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e180bbf-01dd-4df9-87f0-4e660acee001 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:40.520429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a120e8-4977-4fdc-9588-1aea5f05a81f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:40.520857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e180bbf-01dd-4df9-87f0-4e660acee001 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:40.520964 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a120e8-4977-4fdc-9588-1aea5f05a81f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:40.521142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e180bbf-01dd-4df9-87f0-4e660acee001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:40.524389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a120e8-4977-4fdc-9588-1aea5f05a81f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:40.542460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 542/1072] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:43:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:40.549718 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a120e8-4977-4fdc-9588-1aea5f05a81f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:40.554876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-2e180bbf-01dd-4df9-87f0-4e660acee001 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:40.556214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2e180bbf-01dd-4df9-87f0-4e660acee001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:40.657976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:40.668269 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 531/1073] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:40.744422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a120e8-4977-4fdc-9588-1aea5f05a81f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:40.791685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5225c4b-6e6c-4d2b-8084-1e6b71691452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:40.792650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5225c4b-6e6c-4d2b-8084-1e6b71691452 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:40.792927 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5225c4b-6e6c-4d2b-8084-1e6b71691452 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:40.793206 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5225c4b-6e6c-4d2b-8084-1e6b71691452 None admin] PATH_INFO: `/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/ca56dd7216fb4946946b2801e144fec2/roles/de7f9ded8f5f43ff9ba38752e2139702` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:40.795985 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5225c4b-6e6c-4d2b-8084-1e6b71691452 None admin] RBAC: Authorizing `identity:create_grant(project_id=fce6ee19f58b4a468c9a5d169e29a018, user_id=ca56dd7216fb4946946b2801e144fec2, role_id=de7f9ded8f5f43ff9ba38752e2139702)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:40.797917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5225c4b-6e6c-4d2b-8084-1e6b71691452 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:40.820996 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 543/1074] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:43:40 2022] PUT /identity/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/ca56dd7216fb4946946b2801e144fec2/roles/de7f9ded8f5f43ff9ba38752e2139702 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:43:40.935561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e180bbf-01dd-4df9-87f0-4e660acee001 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:41.051412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:41.068990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e180bbf-01dd-4df9-87f0-4e660acee001 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:41.134688 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bebf54-26fb-421d-8d69-66d95c2d31ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:41.136197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bebf54-26fb-421d-8d69-66d95c2d31ca None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:41.136921 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bebf54-26fb-421d-8d69-66d95c2d31ca None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:41.137690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bebf54-26fb-421d-8d69-66d95c2d31ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:41.157823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bebf54-26fb-421d-8d69-66d95c2d31ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:43:41.168182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-03bebf54-26fb-421d-8d69-66d95c2d31ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:43:41.168182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:41.168182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:41.168182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:41.168182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:41.168182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:41.168182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:41.168182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:41.168182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:41.168182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:41.168182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:41.168182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:41.168182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:43:41.168182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:43:41.168182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:43:41.168182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:43:41.169914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:43:41.169914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:43:41.169914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:43:41.169914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:43:41.169914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:41.169914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:41.169914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:41.169914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:41.169914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:41.169914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:41.169914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:41.169914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:41.169914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:41.169914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:41.169914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:41.169914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:41.171528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:41.171528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:41.171528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:41.171528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:41.171528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:43:41.171528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:43:41.171528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:43:41.171528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:43:41.171528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:43:41.171528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:43:41.171528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:43:41.171528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:41.174304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 532/1075] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:43:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:41.179726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5225c4b-6e6c-4d2b-8084-1e6b71691452 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:41.349064 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5225c4b-6e6c-4d2b-8084-1e6b71691452 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:41.377357 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:41.404930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeedfb7-858b-4242-a301-4fbf1da10d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:41.405552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeedfb7-858b-4242-a301-4fbf1da10d6e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:41.405827 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeedfb7-858b-4242-a301-4fbf1da10d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:41.406091 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeedfb7-858b-4242-a301-4fbf1da10d6e None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:41.406768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeedfb7-858b-4242-a301-4fbf1da10d6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:41.407062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeedfb7-858b-4242-a301-4fbf1da10d6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:41.408466 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeedfb7-858b-4242-a301-4fbf1da10d6e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:41.423417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 544/1076] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:43:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:41.425874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bebf54-26fb-421d-8d69-66d95c2d31ca None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:41.568707 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b091367-c3ba-4961-90b0-55efe23e5dd0 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:41.568968 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b091367-c3ba-4961-90b0-55efe23e5dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:41.569254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b091367-c3ba-4961-90b0-55efe23e5dd0 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:41.571716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 545/1077] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:43:41.576967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf25b14-29ba-478a-8208-e1173f94d997 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:41.577284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf25b14-29ba-478a-8208-e1173f94d997 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:41.577519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf25b14-29ba-478a-8208-e1173f94d997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:41.598805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2cf25b14-29ba-478a-8208-e1173f94d997 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:41.600152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2cf25b14-29ba-478a-8208-e1173f94d997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:41.605903 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bebf54-26fb-421d-8d69-66d95c2d31ca None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:41.657540 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629330c-758f-430b-b6aa-6c6e9081cf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:41.658623 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629330c-758f-430b-b6aa-6c6e9081cf66 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:41.659108 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629330c-758f-430b-b6aa-6c6e9081cf66 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:41.659386 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629330c-758f-430b-b6aa-6c6e9081cf66 None admin] PATH_INFO: `/v3/users/6719b6bc9b4c4afe862a2f1ffe359a88` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:41.660828 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629330c-758f-430b-b6aa-6c6e9081cf66 None admin] RBAC: Authorizing `identity:get_user(user_id=6719b6bc9b4c4afe862a2f1ffe359a88)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:41.662721 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629330c-758f-430b-b6aa-6c6e9081cf66 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:41.670653 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 533/1078] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:43:41 2022] GET /identity/v3/users/6719b6bc9b4c4afe862a2f1ffe359a88 => generated 285 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:41.673187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629330c-758f-430b-b6aa-6c6e9081cf66 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:41.691847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 546/1079] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:43:41.701901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:41.838238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629330c-758f-430b-b6aa-6c6e9081cf66 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:41.895312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a708fd-108a-48b2-b850-c476e5ebde3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:41.896183 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a708fd-108a-48b2-b850-c476e5ebde3c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:41.896608 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a708fd-108a-48b2-b850-c476e5ebde3c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:41.896906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a708fd-108a-48b2-b850-c476e5ebde3c None admin] PATH_INFO: `/v3/projects/08e8989146554b8c9c2b6c8d31ccaa10` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:41.898892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a708fd-108a-48b2-b850-c476e5ebde3c None admin] RBAC: Authorizing `identity:get_project(project_id=08e8989146554b8c9c2b6c8d31ccaa10)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:41.900986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a708fd-108a-48b2-b850-c476e5ebde3c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:41.903284 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 534/1080] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:43:41 2022] GET /identity/v3/projects/08e8989146554b8c9c2b6c8d31ccaa10 => generated 311 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:41.911294 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf25b14-29ba-478a-8208-e1173f94d997 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:41.984497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a708fd-108a-48b2-b850-c476e5ebde3c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:42.065538 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf25b14-29ba-478a-8208-e1173f94d997 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:42.088281 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:42.127617 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdd0c0-d778-40d9-926b-8e18cc1ca90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:42.128546 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdd0c0-d778-40d9-926b-8e18cc1ca90e None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:42.128827 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdd0c0-d778-40d9-926b-8e18cc1ca90e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:42.129243 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdd0c0-d778-40d9-926b-8e18cc1ca90e None admin] PATH_INFO: `/v3/projects/08e8989146554b8c9c2b6c8d31ccaa10/users/6719b6bc9b4c4afe862a2f1ffe359a88/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:42.133189 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abdd0c0-d778-40d9-926b-8e18cc1ca90e None admin] RBAC: Authorizing `identity:create_grant(project_id=08e8989146554b8c9c2b6c8d31ccaa10, user_id=6719b6bc9b4c4afe862a2f1ffe359a88, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:42.135674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abdd0c0-d778-40d9-926b-8e18cc1ca90e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:42.136401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:42.152847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 547/1081] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:43:41 2022] PUT /identity/v3/projects/08e8989146554b8c9c2b6c8d31ccaa10/users/6719b6bc9b4c4afe862a2f1ffe359a88/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:43:42.164410 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a708fd-108a-48b2-b850-c476e5ebde3c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:42.208085 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86914eb0-dd3b-4238-9e82-372b5d6e9580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:42.208739 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86914eb0-dd3b-4238-9e82-372b5d6e9580 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:42.209028 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86914eb0-dd3b-4238-9e82-372b5d6e9580 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:42.209304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86914eb0-dd3b-4238-9e82-372b5d6e9580 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:42.290826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86914eb0-dd3b-4238-9e82-372b5d6e9580 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:42.292350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86914eb0-dd3b-4238-9e82-372b5d6e9580 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:42.395614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:42.420169 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 535/1082] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:43:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 08:43:42.429643 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdd0c0-d778-40d9-926b-8e18cc1ca90e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:42.614959 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdd0c0-d778-40d9-926b-8e18cc1ca90e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:42.682333 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d400b4-a3b5-4c35-9f21-e71fc88c32ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:42.683244 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d400b4-a3b5-4c35-9f21-e71fc88c32ba None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:42.683720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d400b4-a3b5-4c35-9f21-e71fc88c32ba None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:42.684218 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d400b4-a3b5-4c35-9f21-e71fc88c32ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:42.706821 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d400b4-a3b5-4c35-9f21-e71fc88c32ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:42.709776 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d400b4-a3b5-4c35-9f21-e71fc88c32ba None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:42.738102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d5d400b4-a3b5-4c35-9f21-e71fc88c32ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:42.738102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:42.738102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:42.738102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:42.738102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:42.738102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:42.738102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:42.738102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:42.738102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:42.738102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:42.738102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:42.738102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:42.738102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:43:42.738102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:43:42.738102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:43:42.738102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:43:42.739974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:42.739974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:42.739974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:42.739974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:42.739974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:42.739974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:42.739974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:42.739974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:42.739974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:42.739974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:42.739974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:42.739974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:42.739974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:42.739974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:42.739974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:42.739974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:42.739974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:43:42.741545 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:43:42.741545 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:42.741545 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:42.748150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 548/1083] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:42.749646 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86914eb0-dd3b-4238-9e82-372b5d6e9580 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:42.915173 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86914eb0-dd3b-4238-9e82-372b5d6e9580 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:42.954996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7379e-8225-48b8-afe5-abbb0ac6ba02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:42.955707 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7379e-8225-48b8-afe5-abbb0ac6ba02 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:42.956125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7379e-8225-48b8-afe5-abbb0ac6ba02 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:42.956527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7379e-8225-48b8-afe5-abbb0ac6ba02 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:42.957305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd7379e-8225-48b8-afe5-abbb0ac6ba02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:42.957766 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd7379e-8225-48b8-afe5-abbb0ac6ba02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:42.959284 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd7379e-8225-48b8-afe5-abbb0ac6ba02 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:42.973688 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 536/1084] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:43.163021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d3974-1051-420b-a994-1358860a5b0e None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:43.163289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d3974-1051-420b-a994-1358860a5b0e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:43.163721 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d3974-1051-420b-a994-1358860a5b0e None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:43.165246 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 549/1085] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:43.171174 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec80a8-1376-4576-af86-dfc73187aa9f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:43.171667 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec80a8-1376-4576-af86-dfc73187aa9f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:43.171747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec80a8-1376-4576-af86-dfc73187aa9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:43.193891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-6fec80a8-1376-4576-af86-dfc73187aa9f None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:43.195936 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6fec80a8-1376-4576-af86-dfc73187aa9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:43.255907 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:43.299690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 537/1086] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:43.577701 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d3974-1051-420b-a994-1358860a5b0e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:43.607374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:43.744361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d3974-1051-420b-a994-1358860a5b0e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:43.785583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fef738-5dd1-4d59-a884-0279624dda88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:43.786318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fef738-5dd1-4d59-a884-0279624dda88 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:43.786680 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fef738-5dd1-4d59-a884-0279624dda88 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:43.787019 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fef738-5dd1-4d59-a884-0279624dda88 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:43.800935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fef738-5dd1-4d59-a884-0279624dda88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:43:43.806710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f2fef738-5dd1-4d59-a884-0279624dda88 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:43:43.806710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:43.806710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:43.806710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:43.806710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:43.806710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:43.806710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:43.806710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:43.806710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:43.806710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:43.806710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:43.806710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:43.806710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:43:43.806710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:43:43.806710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:43:43.806710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:43:43.808835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:43:43.808835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:43:43.808835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:43:43.808835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:43:43.808835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:43.808835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:43.808835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:43.808835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:43.808835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:43.808835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:43.808835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:43.808835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:43.808835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:43.808835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:43.808835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:43.808835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:43.810687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:43.810687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:43.810687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:43.810687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:43.810687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:43:43.810687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:43:43.810687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:43:43.810687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:43:43.810687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:43:43.810687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:43:43.810687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 08:43:43.810687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:43.810687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 550/1087] 10.222.0.67 () {56 vars in 1120 bytes} [Wed Aug 24 08:43:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:43.814017 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec80a8-1376-4576-af86-dfc73187aa9f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:43.923330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec80a8-1376-4576-af86-dfc73187aa9f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:43.973368 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40668c9a-fd0b-46f9-9384-4aa59d189b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:43.974456 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40668c9a-fd0b-46f9-9384-4aa59d189b64 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:43.974984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40668c9a-fd0b-46f9-9384-4aa59d189b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:43.975495 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40668c9a-fd0b-46f9-9384-4aa59d189b64 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:43.977954 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40668c9a-fd0b-46f9-9384-4aa59d189b64 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:43.978519 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40668c9a-fd0b-46f9-9384-4aa59d189b64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:43.981267 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40668c9a-fd0b-46f9-9384-4aa59d189b64 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:44.000707 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 538/1088] 10.222.0.67 () {56 vars in 1115 bytes} [Wed Aug 24 08:43:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:44.001957 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fef738-5dd1-4d59-a884-0279624dda88 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:44.093970 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:44.114346 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fef738-5dd1-4d59-a884-0279624dda88 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:44.166259 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1f93c-536b-4b1f-b07d-9e6683603670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:44.166912 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1f93c-536b-4b1f-b07d-9e6683603670 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:44.167187 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1f93c-536b-4b1f-b07d-9e6683603670 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:44.167555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1f93c-536b-4b1f-b07d-9e6683603670 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:44.180349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1f93c-536b-4b1f-b07d-9e6683603670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:44.181738 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1f93c-536b-4b1f-b07d-9e6683603670 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:44.197450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-75d1f93c-536b-4b1f-b07d-9e6683603670 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:44.197450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:44.197450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:44.197450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:44.197450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:44.197450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:44.197450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:44.197450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:44.197450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:44.197450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:44.197450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:44.197450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:44.197450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:43:44.197450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:43:44.197450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:43:44.197450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:43:44.198836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:44.198836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:44.198836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:44.198836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:44.198836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:44.198836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:44.198836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:44.198836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:44.198836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:44.198836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:44.198836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:44.198836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:44.198836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:44.198836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:44.198836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:44.198836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:44.198836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:43:44.200717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:43:44.200717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:44.200717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:44.200717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 551/1089] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:43:44.204753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40668c9a-fd0b-46f9-9384-4aa59d189b64 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:44.241883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e1d15-4f14-4d7d-a089-76aa4112d0f2 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:44.242168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e1d15-4f14-4d7d-a089-76aa4112d0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:44.242445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e1d15-4f14-4d7d-a089-76aa4112d0f2 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:44.243809 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 552/1090] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:43:44.252606 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a275ed-6aee-4c5d-93a6-314247078c1a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:44.252916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a275ed-6aee-4c5d-93a6-314247078c1a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:44.253245 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a275ed-6aee-4c5d-93a6-314247078c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:44.256470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:44.274908 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-73a275ed-6aee-4c5d-93a6-314247078c1a None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:44.276163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-73a275ed-6aee-4c5d-93a6-314247078c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:44.328894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40668c9a-fd0b-46f9-9384-4aa59d189b64 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:44.368143 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 553/1091] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:43:44.392610 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4faa7-f3dd-48ce-89e1-6469cd9f147f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:44.393694 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4faa7-f3dd-48ce-89e1-6469cd9f147f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:44.394211 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4faa7-f3dd-48ce-89e1-6469cd9f147f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:44.394724 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4faa7-f3dd-48ce-89e1-6469cd9f147f None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:44.395948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4faa7-f3dd-48ce-89e1-6469cd9f147f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:44.396519 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4faa7-f3dd-48ce-89e1-6469cd9f147f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:44.399517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4faa7-f3dd-48ce-89e1-6469cd9f147f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:44.420939 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 539/1092] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:44.426751 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a275ed-6aee-4c5d-93a6-314247078c1a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:44.494289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:44.536206 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a275ed-6aee-4c5d-93a6-314247078c1a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:44.578547 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d3a12-08ee-4fc7-ae6e-720dcf79a5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:44.579596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d3a12-08ee-4fc7-ae6e-720dcf79a5ca None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:44.579596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d3a12-08ee-4fc7-ae6e-720dcf79a5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:44.579860 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d3a12-08ee-4fc7-ae6e-720dcf79a5ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:44.587643 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d3a12-08ee-4fc7-ae6e-720dcf79a5ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:44.589131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d3a12-08ee-4fc7-ae6e-720dcf79a5ca None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:44.602542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d52d3a12-08ee-4fc7-ae6e-720dcf79a5ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:43:44.602542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:44.602542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:44.602542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:44.602542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:44.602542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:44.602542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:44.602542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:44.602542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:44.602542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:44.602542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:44.602542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:44.602542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:43:44.602542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:43:44.602542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:43:44.602542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:43:44.603752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:44.603752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:44.603752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:43:44.603752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:44.603752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:43:44.603752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:43:44.603752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:44.603752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:44.603752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:44.603752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:44.603752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:44.603752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:44.603752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:44.603752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:44.603752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:44.603752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:44.603752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:44.603752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:44.604757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:44.604757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:44.604757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:43:44.604757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:43:44.604757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:43:44.604757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:43:44.604757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:43:44.604757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:43:44.604757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 08:43:44.604757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:44.604757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 554/1093] 10.222.0.67 () {56 vars in 1093 bytes} [Wed Aug 24 08:43:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:43:44.612364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4faa7-f3dd-48ce-89e1-6469cd9f147f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:44.636520 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d3a12-08ee-4fc7-ae6e-720dcf79a5ca None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:44.763784 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:44.767805 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4faa7-f3dd-48ce-89e1-6469cd9f147f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:44.773776 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d3a12-08ee-4fc7-ae6e-720dcf79a5ca None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:44.804113 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:44.810542 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440b038-b0f2-4cec-9241-5db661deedd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:44.811520 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440b038-b0f2-4cec-9241-5db661deedd0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:44.811937 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440b038-b0f2-4cec-9241-5db661deedd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:44.812500 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440b038-b0f2-4cec-9241-5db661deedd0 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:44.813360 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440b038-b0f2-4cec-9241-5db661deedd0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:44.813796 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440b038-b0f2-4cec-9241-5db661deedd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:44.815673 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440b038-b0f2-4cec-9241-5db661deedd0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:44.820530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293881c6-f2a1-4178-a5d9-dfebdc58834f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:44.821189 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293881c6-f2a1-4178-a5d9-dfebdc58834f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:44.821474 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293881c6-f2a1-4178-a5d9-dfebdc58834f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:44.821750 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293881c6-f2a1-4178-a5d9-dfebdc58834f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:44.836561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 540/1094] 10.222.0.67 () {56 vars in 1133 bytes} [Wed Aug 24 08:43:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:44.837654 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293881c6-f2a1-4178-a5d9-dfebdc58834f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:43:44.842675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-293881c6-f2a1-4178-a5d9-dfebdc58834f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:43:44.842675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:44.842675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:44.842675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:44.842675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:44.842675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:44.842675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:44.842675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:44.842675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:44.842675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:44.842675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:44.842675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:44.842675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:43:44.842675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:43:44.842675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:43:44.842675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:43:44.844574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:43:44.844574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:43:44.844574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:43:44.844574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:43:44.844574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:44.844574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:44.844574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:44.844574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:44.844574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:44.844574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:44.844574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:44.844574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:44.844574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:44.844574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:44.844574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:44.844574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:44.845685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:44.845685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:44.845685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:44.845685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:44.845685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:43:44.845685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:43:44.845685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:43:44.845685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:43:44.845685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:43:44.845685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:43:44.845685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:43:44.845685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:43:44.845685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 555/1095] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:43:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:43:44.846572 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440b038-b0f2-4cec-9241-5db661deedd0 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:44.849617 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293881c6-f2a1-4178-a5d9-dfebdc58834f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:44.959985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440b038-b0f2-4cec-9241-5db661deedd0 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:44.969650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293881c6-f2a1-4178-a5d9-dfebdc58834f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:44.998935 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5dc457-b33b-4f7c-a76b-b74a2b443b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:44.999575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dc457-b33b-4f7c-a76b-b74a2b443b14 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:44.999843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dc457-b33b-4f7c-a76b-b74a2b443b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:45.000139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dc457-b33b-4f7c-a76b-b74a2b443b14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:45.006479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a384f25-e73f-4516-bd58-f437d2d1a1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:45.007105 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a384f25-e73f-4516-bd58-f437d2d1a1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:45.007568 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a384f25-e73f-4516-bd58-f437d2d1a1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:45.007841 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a384f25-e73f-4516-bd58-f437d2d1a1b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:45.008557 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a384f25-e73f-4516-bd58-f437d2d1a1b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:45.008862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a384f25-e73f-4516-bd58-f437d2d1a1b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:45.010247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a384f25-e73f-4516-bd58-f437d2d1a1b5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:45.022430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 556/1096] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:43:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:45.030385 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a384f25-e73f-4516-bd58-f437d2d1a1b5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:45.036687 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:45.082996 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:45.083582 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5dc457-b33b-4f7c-a76b-b74a2b443b14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:45.085084 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5dc457-b33b-4f7c-a76b-b74a2b443b14 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:45.168000 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a384f25-e73f-4516-bd58-f437d2d1a1b5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:45.200676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 541/1097] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:43:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:43:45.206474 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5dc457-b33b-4f7c-a76b-b74a2b443b14 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:45.214678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01aaaee-d45b-439d-a440-d8eb8683f30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:45.215324 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01aaaee-d45b-439d-a440-d8eb8683f30f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:45.215597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01aaaee-d45b-439d-a440-d8eb8683f30f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:45.215874 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01aaaee-d45b-439d-a440-d8eb8683f30f None admin] PATH_INFO: `/v3/users/6719b6bc9b4c4afe862a2f1ffe359a88` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:45.217085 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01aaaee-d45b-439d-a440-d8eb8683f30f None admin] RBAC: Authorizing `identity:get_user(user_id=6719b6bc9b4c4afe862a2f1ffe359a88)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:45.218526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01aaaee-d45b-439d-a440-d8eb8683f30f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:45.220208 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 557/1098] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:43:45 2022] GET /identity/v3/users/6719b6bc9b4c4afe862a2f1ffe359a88 => generated 285 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:45.227079 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01aaaee-d45b-439d-a440-d8eb8683f30f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:45.283589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:45.333672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5dc457-b33b-4f7c-a76b-b74a2b443b14 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:45.364855 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01aaaee-d45b-439d-a440-d8eb8683f30f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:45.374922 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2810b85-fa8e-428d-83da-58f2c8c2d77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:45.376220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2810b85-fa8e-428d-83da-58f2c8c2d77f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:45.376636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2810b85-fa8e-428d-83da-58f2c8c2d77f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:45.376997 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2810b85-fa8e-428d-83da-58f2c8c2d77f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:45.382550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:45.395737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2810b85-fa8e-428d-83da-58f2c8c2d77f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:45.397454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2810b85-fa8e-428d-83da-58f2c8c2d77f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:45.415396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-d2810b85-fa8e-428d-83da-58f2c8c2d77f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:45.415396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:45.415396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:45.415396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:45.415396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:45.415396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:45.415396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:45.415396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:45.415396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:45.415396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:45.415396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:45.415396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:45.415396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:43:45.415396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:43:45.415396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:43:45.415396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:43:45.420219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:45.420219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:45.420219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:45.420219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:45.420219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:45.420219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:45.420219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:45.420219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:45.420219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:45.420219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:45.420219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:45.420219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:45.420219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:45.420219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:45.420219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:45.420219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:45.420219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:43:45.422394 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:43:45.422394 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:43:45.422394 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:45.422394 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 542/1099] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:43:45.422866 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f14a5-0fe4-4771-ad87-10d0871e2aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:45.422866 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f14a5-0fe4-4771-ad87-10d0871e2aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:45.423153 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2810b85-fa8e-428d-83da-58f2c8c2d77f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:45.423291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f14a5-0fe4-4771-ad87-10d0871e2aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:45.423291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f14a5-0fe4-4771-ad87-10d0871e2aa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:45.518514 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f14a5-0fe4-4771-ad87-10d0871e2aa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:45.519984 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f14a5-0fe4-4771-ad87-10d0871e2aa4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:45.545200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2810b85-fa8e-428d-83da-58f2c8c2d77f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:45.559011 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:45.603568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf60715-319e-4976-8979-4fbce8d62259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:45.604617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf60715-319e-4976-8979-4fbce8d62259 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:45.605119 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf60715-319e-4976-8979-4fbce8d62259 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:45.605609 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf60715-319e-4976-8979-4fbce8d62259 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:45.606826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf60715-319e-4976-8979-4fbce8d62259 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:45.607366 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf60715-319e-4976-8979-4fbce8d62259 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:45.610050 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf60715-319e-4976-8979-4fbce8d62259 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:45.631257 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 543/1100] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:45.637693 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf60715-319e-4976-8979-4fbce8d62259 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:45.643020 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 558/1101] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:43:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 08:43:45.652576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f14a5-0fe4-4771-ad87-10d0871e2aa4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:45.680127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:45.762205 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf60715-319e-4976-8979-4fbce8d62259 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:45.800688 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af45494-4b42-4847-a00a-b65bcb6202b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:45.801240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af45494-4b42-4847-a00a-b65bcb6202b3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:45.801525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af45494-4b42-4847-a00a-b65bcb6202b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:45.801796 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af45494-4b42-4847-a00a-b65bcb6202b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:45.803900 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f14a5-0fe4-4771-ad87-10d0871e2aa4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:45.810835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af45494-4b42-4847-a00a-b65bcb6202b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:45.812323 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af45494-4b42-4847-a00a-b65bcb6202b3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:45.824955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-7af45494-4b42-4847-a00a-b65bcb6202b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:45.824955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:43:45.824955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:43:45.824955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:43:45.824955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:43:45.824955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:43:45.824955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:43:45.824955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:43:45.824955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:43:45.824955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:43:45.824955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:43:45.824955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:43:45.824955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:43:45.824955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:43:45.824955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:43:45.826082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:43:45.826082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:43:45.826082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:43:45.826082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:43:45.826082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:43:45.826082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:43:45.826082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:43:45.826082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:43:45.826082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:43:45.826082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:43:45.826082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:43:45.826082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:43:45.826082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:43:45.826082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:43:45.826082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:43:45.826082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:43:45.826082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:43:45.826970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:43:45.826970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:43:45.826970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:43:45.826970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:43:45.826970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:43:45.826970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:43:45.826970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:43:45.826970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:43:45.826970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 544/1102] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:43:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:43:45.832080 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af45494-4b42-4847-a00a-b65bcb6202b3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:45.843597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7a69d-01c0-4fac-bfb3-83023dd47767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:45.844374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7a69d-01c0-4fac-bfb3-83023dd47767 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:45.844656 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7a69d-01c0-4fac-bfb3-83023dd47767 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:45.844923 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7a69d-01c0-4fac-bfb3-83023dd47767 None admin] PATH_INFO: `/v3/projects/08e8989146554b8c9c2b6c8d31ccaa10` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:45.846064 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a7a69d-01c0-4fac-bfb3-83023dd47767 None admin] RBAC: Authorizing `identity:get_project(project_id=08e8989146554b8c9c2b6c8d31ccaa10)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:45.847501 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a7a69d-01c0-4fac-bfb3-83023dd47767 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:45.849223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 559/1103] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:43:45 2022] GET /identity/v3/projects/08e8989146554b8c9c2b6c8d31ccaa10 => generated 311 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:45.855775 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7a69d-01c0-4fac-bfb3-83023dd47767 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:45.866415 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:45.968668 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:45.975409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af45494-4b42-4847-a00a-b65bcb6202b3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:45.995134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7a69d-01c0-4fac-bfb3-83023dd47767 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:46.015368 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f69eb-71b3-402e-9693-b9a035d36661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:46.016004 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f69eb-71b3-402e-9693-b9a035d36661 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:46.016312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f69eb-71b3-402e-9693-b9a035d36661 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:46.016581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f69eb-71b3-402e-9693-b9a035d36661 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:46.017281 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f69eb-71b3-402e-9693-b9a035d36661 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:46.017585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f69eb-71b3-402e-9693-b9a035d36661 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:46.019016 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f69eb-71b3-402e-9693-b9a035d36661 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:46.031944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 545/1104] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:43:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:46.038452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f69eb-71b3-402e-9693-b9a035d36661 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:46.048138 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619ddfa-5053-4acb-a564-e83aca3ad74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:46.048876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619ddfa-5053-4acb-a564-e83aca3ad74b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:46.050634 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619ddfa-5053-4acb-a564-e83aca3ad74b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:46.050634 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619ddfa-5053-4acb-a564-e83aca3ad74b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:46.050634 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619ddfa-5053-4acb-a564-e83aca3ad74b None admin] RBAC: Adding query filter params (role.id=b13ffa72344c4856901b1ba003e273fd, scope.project.id=08e8989146554b8c9c2b6c8d31ccaa10, user.id=6719b6bc9b4c4afe862a2f1ffe359a88) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:46.050634 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619ddfa-5053-4acb-a564-e83aca3ad74b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:46.051806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619ddfa-5053-4acb-a564-e83aca3ad74b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:46.072132 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 560/1105] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:43:45 2022] GET /identity/v3/role_assignments?user.id=6719b6bc9b4c4afe862a2f1ffe359a88&scope.project.id=08e8989146554b8c9c2b6c8d31ccaa10&role.id=b13ffa72344c4856901b1ba003e273fd => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:46.189857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:46.207491 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f69eb-71b3-402e-9693-b9a035d36661 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:46.268590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4db1f-62f0-4bc0-99c6-3449fd5bdcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:46.269818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4db1f-62f0-4bc0-99c6-3449fd5bdcdf None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:46.270272 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4db1f-62f0-4bc0-99c6-3449fd5bdcdf None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:46.270698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4db1f-62f0-4bc0-99c6-3449fd5bdcdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:46.271923 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4db1f-62f0-4bc0-99c6-3449fd5bdcdf None admin] RBAC: Adding query filter params (role.id=de7f9ded8f5f43ff9ba38752e2139702, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=ca56dd7216fb4946946b2801e144fec2) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:46.272420 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4db1f-62f0-4bc0-99c6-3449fd5bdcdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:46.275221 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4db1f-62f0-4bc0-99c6-3449fd5bdcdf None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:46.284653 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 546/1106] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:43:46 2022] GET /identity/v3/role_assignments?user.id=ca56dd7216fb4946946b2801e144fec2&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=de7f9ded8f5f43ff9ba38752e2139702 => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:43:48.176915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361de0a-6742-4239-8628-eefa79f738ca None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:48.177459 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361de0a-6742-4239-8628-eefa79f738ca None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:48.177459 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361de0a-6742-4239-8628-eefa79f738ca None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:48.179084 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 561/1107] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:48.185537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284f2fd-903f-4569-82a3-d4e0da00fbd9 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:48.186338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284f2fd-903f-4569-82a3-d4e0da00fbd9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:48.186338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284f2fd-903f-4569-82a3-d4e0da00fbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:48.205461 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d284f2fd-903f-4569-82a3-d4e0da00fbd9 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:48.206653 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d284f2fd-903f-4569-82a3-d4e0da00fbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:48.280549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 547/1108] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:48.550555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361de0a-6742-4239-8628-eefa79f738ca None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:48.712004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361de0a-6742-4239-8628-eefa79f738ca None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:48.776002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97bd50-43d0-435a-969f-d128cf84aa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:48.777150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97bd50-43d0-435a-969f-d128cf84aa67 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:48.777663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97bd50-43d0-435a-969f-d128cf84aa67 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:48.778169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97bd50-43d0-435a-969f-d128cf84aa67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:48.780207 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97bd50-43d0-435a-969f-d128cf84aa67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:48.783022 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97bd50-43d0-435a-969f-d128cf84aa67 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:48.786124 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 562/1109] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:43:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:48.792335 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284f2fd-903f-4569-82a3-d4e0da00fbd9 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:48.883743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:48.909538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284f2fd-903f-4569-82a3-d4e0da00fbd9 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:48.962687 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1bb89-7367-445f-8f99-4eac7cead9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:48.963299 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1bb89-7367-445f-8f99-4eac7cead9c2 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:48.963573 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1bb89-7367-445f-8f99-4eac7cead9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:48.963838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1bb89-7367-445f-8f99-4eac7cead9c2 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:48.964527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1bb89-7367-445f-8f99-4eac7cead9c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:48.965951 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1bb89-7367-445f-8f99-4eac7cead9c2 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:48.992458 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 548/1110] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 24 08:43:48 2022] POST /identity/v3/users => generated 286 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:43:50.971881 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b0f3c-9045-4691-ba7b-5210f3866a4d None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:50.972504 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b0f3c-9045-4691-ba7b-5210f3866a4d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:50.972504 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b0f3c-9045-4691-ba7b-5210f3866a4d None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:50.973826 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 563/1111] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:50.980698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f71072-b80a-4169-bdc3-c8c12fdc3a09 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:50.981228 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f71072-b80a-4169-bdc3-c8c12fdc3a09 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:50.981228 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f71072-b80a-4169-bdc3-c8c12fdc3a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:51.001124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f8f71072-b80a-4169-bdc3-c8c12fdc3a09 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:51.002251 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f8f71072-b80a-4169-bdc3-c8c12fdc3a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:51.083627 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 549/1112] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:51.350286 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b0f3c-9045-4691-ba7b-5210f3866a4d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:51.367143 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:51.460413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b0f3c-9045-4691-ba7b-5210f3866a4d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:51.505135 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75348db3-ba71-4dc0-bbd1-5255c84f8257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:51.505762 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75348db3-ba71-4dc0-bbd1-5255c84f8257 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:51.506034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75348db3-ba71-4dc0-bbd1-5255c84f8257 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:51.506297 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75348db3-ba71-4dc0-bbd1-5255c84f8257 None admin] PATH_INFO: `/v3/roles/5915b1954b514151ab17ff5d531036e4` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:51.507692 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75348db3-ba71-4dc0-bbd1-5255c84f8257 None admin] RBAC: Authorizing `identity:get_role(role_id=5915b1954b514151ab17ff5d531036e4)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:51.509086 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75348db3-ba71-4dc0-bbd1-5255c84f8257 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:51.510734 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 564/1113] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:43:51 2022] GET /identity/v3/roles/5915b1954b514151ab17ff5d531036e4 => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:51.516651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f71072-b80a-4169-bdc3-c8c12fdc3a09 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:51.586870 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:51.622167 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f71072-b80a-4169-bdc3-c8c12fdc3a09 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:51.657948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d3bd6-cc60-4df3-86a8-d75442587503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:51.658587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d3bd6-cc60-4df3-86a8-d75442587503 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:51.658868 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d3bd6-cc60-4df3-86a8-d75442587503 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:51.659141 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d3bd6-cc60-4df3-86a8-d75442587503 None admin] PATH_INFO: `/v3/users/9640cc60d24248eeb362bc21d66cbf8d` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:51.674274 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d3bd6-cc60-4df3-86a8-d75442587503 None admin] RBAC: Authorizing `identity:get_user(user_id=9640cc60d24248eeb362bc21d66cbf8d)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:51.675726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d3bd6-cc60-4df3-86a8-d75442587503 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:51.677517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 550/1114] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:43:51 2022] GET /identity/v3/users/9640cc60d24248eeb362bc21d66cbf8d => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:51.684070 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75348db3-ba71-4dc0-bbd1-5255c84f8257 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:51.797540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:51.817134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75348db3-ba71-4dc0-bbd1-5255c84f8257 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:51.854874 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c263d6-920d-4f3f-bbc3-aa45485806c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:51.855470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c263d6-920d-4f3f-bbc3-aa45485806c0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:51.855744 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c263d6-920d-4f3f-bbc3-aa45485806c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:51.856009 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c263d6-920d-4f3f-bbc3-aa45485806c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:51.931849 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c263d6-920d-4f3f-bbc3-aa45485806c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:51.933482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c263d6-920d-4f3f-bbc3-aa45485806c0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:52.021882 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:52.042159 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 565/1115] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:43:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:43:52.050463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d3bd6-cc60-4df3-86a8-d75442587503 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:52.157352 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d3bd6-cc60-4df3-86a8-d75442587503 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:52.186691 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:52.195589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e3622-2de3-47c5-b996-e0f34b545d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:52.196268 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e3622-2de3-47c5-b996-e0f34b545d44 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:52.196549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e3622-2de3-47c5-b996-e0f34b545d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:52.196820 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e3622-2de3-47c5-b996-e0f34b545d44 None admin] PATH_INFO: `/v3/projects/08e8989146554b8c9c2b6c8d31ccaa10` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:52.198028 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e3622-2de3-47c5-b996-e0f34b545d44 None admin] RBAC: Authorizing `identity:get_project(project_id=08e8989146554b8c9c2b6c8d31ccaa10)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:52.199450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e3622-2de3-47c5-b996-e0f34b545d44 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:52.201141 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 551/1116] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:43:52 2022] GET /identity/v3/projects/08e8989146554b8c9c2b6c8d31ccaa10 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:52.207478 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c263d6-920d-4f3f-bbc3-aa45485806c0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:52.318388 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c263d6-920d-4f3f-bbc3-aa45485806c0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:52.354158 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4a399-f4aa-4119-9840-b9ce074534b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:52.354761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4a399-f4aa-4119-9840-b9ce074534b6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:52.355043 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4a399-f4aa-4119-9840-b9ce074534b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:52.355313 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4a399-f4aa-4119-9840-b9ce074534b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:52.356044 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4a399-f4aa-4119-9840-b9ce074534b6 None admin] RBAC: Adding query filter params (role.id=5915b1954b514151ab17ff5d531036e4, scope.project.id=08e8989146554b8c9c2b6c8d31ccaa10, user.id=9640cc60d24248eeb362bc21d66cbf8d) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:52.356383 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4a399-f4aa-4119-9840-b9ce074534b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:52.357815 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4a399-f4aa-4119-9840-b9ce074534b6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:52.363557 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 566/1117] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:43:52 2022] GET /identity/v3/role_assignments?user.id=9640cc60d24248eeb362bc21d66cbf8d&scope.project.id=08e8989146554b8c9c2b6c8d31ccaa10&role.id=5915b1954b514151ab17ff5d531036e4 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:54.307808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdacb70-104d-495e-bf5e-3f93341e2e0d None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:54.308388 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdacb70-104d-495e-bf5e-3f93341e2e0d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:54.308388 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdacb70-104d-495e-bf5e-3f93341e2e0d None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:54.309748 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 552/1118] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:54.314932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec3a2c-b791-4d22-bca8-0ee6cac64fdd None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:54.315357 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec3a2c-b791-4d22-bca8-0ee6cac64fdd None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:54.315449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec3a2c-b791-4d22-bca8-0ee6cac64fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:54.334183 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-68ec3a2c-b791-4d22-bca8-0ee6cac64fdd None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:54.335361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-68ec3a2c-b791-4d22-bca8-0ee6cac64fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:54.381493 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:54.408453 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 567/1119] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:54.691832 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdacb70-104d-495e-bf5e-3f93341e2e0d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:54.798307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdacb70-104d-495e-bf5e-3f93341e2e0d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:54.834173 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a03d0-746c-43b1-bc75-4a51ed321a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:54.834809 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a03d0-746c-43b1-bc75-4a51ed321a9d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:54.835083 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a03d0-746c-43b1-bc75-4a51ed321a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:54.835350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a03d0-746c-43b1-bc75-4a51ed321a9d None admin] PATH_INFO: `/v3/roles/5915b1954b514151ab17ff5d531036e4` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:54.836999 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a03d0-746c-43b1-bc75-4a51ed321a9d None admin] RBAC: Authorizing `identity:get_role(role_id=5915b1954b514151ab17ff5d531036e4)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:54.838354 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a03d0-746c-43b1-bc75-4a51ed321a9d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:54.839704 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 553/1120] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:43:54 2022] GET /identity/v3/roles/5915b1954b514151ab17ff5d531036e4 => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:54.845937 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec3a2c-b791-4d22-bca8-0ee6cac64fdd None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:54.953455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec3a2c-b791-4d22-bca8-0ee6cac64fdd None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:54.987709 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bfe55b-caef-482d-bfaa-e610a4e80b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:54.988381 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bfe55b-caef-482d-bfaa-e610a4e80b90 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:54.988659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bfe55b-caef-482d-bfaa-e610a4e80b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:54.988930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bfe55b-caef-482d-bfaa-e610a4e80b90 None admin] PATH_INFO: `/v3/users/9640cc60d24248eeb362bc21d66cbf8d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:54.990051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bfe55b-caef-482d-bfaa-e610a4e80b90 None admin] RBAC: Authorizing `identity:get_user(user_id=9640cc60d24248eeb362bc21d66cbf8d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:54.991470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bfe55b-caef-482d-bfaa-e610a4e80b90 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:54.994015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 568/1121] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:43:54 2022] GET /identity/v3/users/9640cc60d24248eeb362bc21d66cbf8d => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:54.999971 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a03d0-746c-43b1-bc75-4a51ed321a9d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:55.065252 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:55.106396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a03d0-746c-43b1-bc75-4a51ed321a9d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:55.146784 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fffec1-0731-41e2-9099-69560f4e33e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:55.147415 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fffec1-0731-41e2-9099-69560f4e33e4 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:55.147696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fffec1-0731-41e2-9099-69560f4e33e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:55.147969 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fffec1-0731-41e2-9099-69560f4e33e4 None admin] PATH_INFO: `/v3/projects/08e8989146554b8c9c2b6c8d31ccaa10` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:55.149765 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fffec1-0731-41e2-9099-69560f4e33e4 None admin] RBAC: Authorizing `identity:get_project(project_id=08e8989146554b8c9c2b6c8d31ccaa10)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:55.151294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fffec1-0731-41e2-9099-69560f4e33e4 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:55.152992 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 554/1122] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:43:54 2022] GET /identity/v3/projects/08e8989146554b8c9c2b6c8d31ccaa10 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:55.159349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bfe55b-caef-482d-bfaa-e610a4e80b90 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:55.205877 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:55.264227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bfe55b-caef-482d-bfaa-e610a4e80b90 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:55.299013 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36137e0-2914-4ebc-9fe7-45f793870349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:55.299682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36137e0-2914-4ebc-9fe7-45f793870349 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:55.299964 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36137e0-2914-4ebc-9fe7-45f793870349 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:55.300271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36137e0-2914-4ebc-9fe7-45f793870349 None admin] PATH_INFO: `/v3/projects/08e8989146554b8c9c2b6c8d31ccaa10/users/9640cc60d24248eeb362bc21d66cbf8d/roles/5915b1954b514151ab17ff5d531036e4` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:55.302443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36137e0-2914-4ebc-9fe7-45f793870349 None admin] RBAC: Authorizing `identity:create_grant(project_id=08e8989146554b8c9c2b6c8d31ccaa10, user_id=9640cc60d24248eeb362bc21d66cbf8d, role_id=5915b1954b514151ab17ff5d531036e4)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:55.304134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36137e0-2914-4ebc-9fe7-45f793870349 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:55.317468 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 569/1123] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:43:55 2022] PUT /identity/v3/projects/08e8989146554b8c9c2b6c8d31ccaa10/users/9640cc60d24248eeb362bc21d66cbf8d/roles/5915b1954b514151ab17ff5d531036e4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:43:57.243959 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9af5e6-a8d5-449f-af69-cf36b128e63f None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:57.244522 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9af5e6-a8d5-449f-af69-cf36b128e63f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:57.244645 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9af5e6-a8d5-449f-af69-cf36b128e63f None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:57.245921 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 555/1124] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:43:57.251335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f24a87-7d47-4dc1-8c3e-225a4fc331db None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:57.251739 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f24a87-7d47-4dc1-8c3e-225a4fc331db None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:57.251874 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f24a87-7d47-4dc1-8c3e-225a4fc331db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:57.270524 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-73f24a87-7d47-4dc1-8c3e-225a4fc331db None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:43:57.271690 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-73f24a87-7d47-4dc1-8c3e-225a4fc331db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:43:57.341144 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 570/1125] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:43:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:43:57.616207 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9af5e6-a8d5-449f-af69-cf36b128e63f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:57.717310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9af5e6-a8d5-449f-af69-cf36b128e63f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:57.748576 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:57.754034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b66c66-207e-4c46-93c1-a343ef51db2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:57.754673 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b66c66-207e-4c46-93c1-a343ef51db2d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:57.754957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b66c66-207e-4c46-93c1-a343ef51db2d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:57.755231 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b66c66-207e-4c46-93c1-a343ef51db2d None admin] PATH_INFO: `/v3/roles/5915b1954b514151ab17ff5d531036e4` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:57.756654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b66c66-207e-4c46-93c1-a343ef51db2d None admin] RBAC: Authorizing `identity:get_role(role_id=5915b1954b514151ab17ff5d531036e4)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:57.758129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b66c66-207e-4c46-93c1-a343ef51db2d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:57.759470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 556/1126] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:43:57 2022] GET /identity/v3/roles/5915b1954b514151ab17ff5d531036e4 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:57.765355 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f24a87-7d47-4dc1-8c3e-225a4fc331db None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:57.799297 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:57.866715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f24a87-7d47-4dc1-8c3e-225a4fc331db None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:57.900741 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aabd9a-3855-4bf1-86f5-48eaa2e5a4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:57.901359 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aabd9a-3855-4bf1-86f5-48eaa2e5a4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:57.901631 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aabd9a-3855-4bf1-86f5-48eaa2e5a4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:57.901902 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aabd9a-3855-4bf1-86f5-48eaa2e5a4e6 None admin] PATH_INFO: `/v3/users/9640cc60d24248eeb362bc21d66cbf8d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:57.903003 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aabd9a-3855-4bf1-86f5-48eaa2e5a4e6 None admin] RBAC: Authorizing `identity:get_user(user_id=9640cc60d24248eeb362bc21d66cbf8d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:57.904451 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aabd9a-3855-4bf1-86f5-48eaa2e5a4e6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:57.906022 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 571/1127] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:43:57 2022] GET /identity/v3/users/9640cc60d24248eeb362bc21d66cbf8d => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:57.911854 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b66c66-207e-4c46-93c1-a343ef51db2d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:58.016933 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b66c66-207e-4c46-93c1-a343ef51db2d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:58.053458 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a3928-8189-4ac2-b14e-5462ee3db08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:58.054348 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a3928-8189-4ac2-b14e-5462ee3db08e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:58.055295 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a3928-8189-4ac2-b14e-5462ee3db08e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:58.055569 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a3928-8189-4ac2-b14e-5462ee3db08e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:58.125348 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:58.127449 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a3928-8189-4ac2-b14e-5462ee3db08e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:58.129448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a3928-8189-4ac2-b14e-5462ee3db08e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:58.232317 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 557/1128] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:43:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:43:58.238315 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aabd9a-3855-4bf1-86f5-48eaa2e5a4e6 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:58.341802 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aabd9a-3855-4bf1-86f5-48eaa2e5a4e6 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:58.343377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:58.378647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58ac07-8472-40a6-853b-aab275f6cce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:58.379189 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58ac07-8472-40a6-853b-aab275f6cce0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:58.379467 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58ac07-8472-40a6-853b-aab275f6cce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:58.379734 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58ac07-8472-40a6-853b-aab275f6cce0 None admin] PATH_INFO: `/v3/projects/08e8989146554b8c9c2b6c8d31ccaa10` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:58.380888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be58ac07-8472-40a6-853b-aab275f6cce0 None admin] RBAC: Authorizing `identity:get_project(project_id=08e8989146554b8c9c2b6c8d31ccaa10)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:58.382311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be58ac07-8472-40a6-853b-aab275f6cce0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:58.384904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 572/1129] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:43:58 2022] GET /identity/v3/projects/08e8989146554b8c9c2b6c8d31ccaa10 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:43:58.391135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a3928-8189-4ac2-b14e-5462ee3db08e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:43:58.494274 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:43:58.497266 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a3928-8189-4ac2-b14e-5462ee3db08e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:43:58.533350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab4de0-fac2-4535-abcd-207909f1c78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:43:58.533962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab4de0-fac2-4535-abcd-207909f1c78a None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:43:58.534242 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab4de0-fac2-4535-abcd-207909f1c78a None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:43:58.534512 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab4de0-fac2-4535-abcd-207909f1c78a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:43:58.535238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab4de0-fac2-4535-abcd-207909f1c78a None admin] RBAC: Adding query filter params (role.id=5915b1954b514151ab17ff5d531036e4, scope.project.id=08e8989146554b8c9c2b6c8d31ccaa10, user.id=9640cc60d24248eeb362bc21d66cbf8d) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:43:58.535548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab4de0-fac2-4535-abcd-207909f1c78a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:43:58.537000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab4de0-fac2-4535-abcd-207909f1c78a None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:43:58.543728 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 558/1130] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:43:58 2022] GET /identity/v3/role_assignments?user.id=9640cc60d24248eeb362bc21d66cbf8d&scope.project.id=08e8989146554b8c9c2b6c8d31ccaa10&role.id=5915b1954b514151ab17ff5d531036e4 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:00.518459 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4c13f-1543-48ec-bfd8-45beef40de5c None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:00.519071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4c13f-1543-48ec-bfd8-45beef40de5c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:00.519071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4c13f-1543-48ec-bfd8-45beef40de5c None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:00.520359 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 573/1131] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:44:00.527588 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a81ac-ea8c-48fd-a76c-d12ae6373485 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:00.528120 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a81ac-ea8c-48fd-a76c-d12ae6373485 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:00.528221 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a81ac-ea8c-48fd-a76c-d12ae6373485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:00.548959 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-548a81ac-ea8c-48fd-a76c-d12ae6373485 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:44:00.550147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-548a81ac-ea8c-48fd-a76c-d12ae6373485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:44:00.619747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 559/1132] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:44:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:44:00.896572 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4c13f-1543-48ec-bfd8-45beef40de5c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:01.003571 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4c13f-1543-48ec-bfd8-45beef40de5c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:01.037249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af0354-4baa-4c2b-9813-212fe644475c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:01.037874 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af0354-4baa-4c2b-9813-212fe644475c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:01.038150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af0354-4baa-4c2b-9813-212fe644475c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:01.038418 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af0354-4baa-4c2b-9813-212fe644475c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:01.040027 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af0354-4baa-4c2b-9813-212fe644475c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:01.041456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af0354-4baa-4c2b-9813-212fe644475c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:01.043031 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 574/1133] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:44:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:01.048939 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a81ac-ea8c-48fd-a76c-d12ae6373485 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:01.142150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:01.150148 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a81ac-ea8c-48fd-a76c-d12ae6373485 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:01.183675 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b713921-3204-4336-8f95-bfb30637f654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:01.184306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b713921-3204-4336-8f95-bfb30637f654 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:01.184587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b713921-3204-4336-8f95-bfb30637f654 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:01.184861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b713921-3204-4336-8f95-bfb30637f654 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:01.185500 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b713921-3204-4336-8f95-bfb30637f654 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:01.186918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b713921-3204-4336-8f95-bfb30637f654 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:01.214827 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 560/1134] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 24 08:44:01 2022] POST /identity/v3/projects => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:44:03.169647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fbdd0-36a6-497e-a9ac-70960aca742f None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:03.170205 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fbdd0-36a6-497e-a9ac-70960aca742f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:03.170205 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fbdd0-36a6-497e-a9ac-70960aca742f None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:03.171534 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 575/1135] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:44:03.177502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3d3c0-d17e-4499-a6ad-99b1533c2670 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:03.178028 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3d3c0-d17e-4499-a6ad-99b1533c2670 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:03.178028 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3d3c0-d17e-4499-a6ad-99b1533c2670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:03.197375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-edb3d3c0-d17e-4499-a6ad-99b1533c2670 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:44:03.198644 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-edb3d3c0-d17e-4499-a6ad-99b1533c2670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:44:03.266390 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 561/1136] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:44:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:44:03.536864 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fbdd0-36a6-497e-a9ac-70960aca742f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:03.579627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:03.644724 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fbdd0-36a6-497e-a9ac-70960aca742f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:03.678700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf13087-4d1f-47b0-8498-70b3f42d30ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:03.679572 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf13087-4d1f-47b0-8498-70b3f42d30ec None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:03.680510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf13087-4d1f-47b0-8498-70b3f42d30ec None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:03.680776 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf13087-4d1f-47b0-8498-70b3f42d30ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:03.681972 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf13087-4d1f-47b0-8498-70b3f42d30ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:03.683415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf13087-4d1f-47b0-8498-70b3f42d30ec None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:03.685051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 576/1137] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:44:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:03.690805 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3d3c0-d17e-4499-a6ad-99b1533c2670 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:03.770239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:03.794479 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3d3c0-d17e-4499-a6ad-99b1533c2670 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:03.828550 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d7da8-7308-42e3-afe5-ee941fb635bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:03.829156 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d7da8-7308-42e3-afe5-ee941fb635bd None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:03.829432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d7da8-7308-42e3-afe5-ee941fb635bd None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:03.829697 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d7da8-7308-42e3-afe5-ee941fb635bd None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:03.830343 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d7da8-7308-42e3-afe5-ee941fb635bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:03.831757 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d7da8-7308-42e3-afe5-ee941fb635bd None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:03.854011 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 562/1138] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 24 08:44:03 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:44:05.847645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea7355-97bd-429e-8025-6c9940a454f3 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:05.848229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea7355-97bd-429e-8025-6c9940a454f3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:05.848329 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea7355-97bd-429e-8025-6c9940a454f3 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:05.849617 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 577/1139] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:44:05.855557 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c4698-1b63-4e34-8d91-4e17f5a0bf6e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:05.855979 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c4698-1b63-4e34-8d91-4e17f5a0bf6e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:05.856087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c4698-1b63-4e34-8d91-4e17f5a0bf6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:05.874498 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d29c4698-1b63-4e34-8d91-4e17f5a0bf6e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:44:05.875672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d29c4698-1b63-4e34-8d91-4e17f5a0bf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:44:05.941362 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 563/1140] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:44:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:44:06.205536 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea7355-97bd-429e-8025-6c9940a454f3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:06.312437 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea7355-97bd-429e-8025-6c9940a454f3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:06.319941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:06.346504 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e693d23-4a93-44dd-bd49-06f3e6c8ed76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:06.347136 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e693d23-4a93-44dd-bd49-06f3e6c8ed76 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:06.347409 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e693d23-4a93-44dd-bd49-06f3e6c8ed76 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:06.347682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e693d23-4a93-44dd-bd49-06f3e6c8ed76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:06.358628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e693d23-4a93-44dd-bd49-06f3e6c8ed76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:44:06.363317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-2e693d23-4a93-44dd-bd49-06f3e6c8ed76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:44:06.363317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:44:06.363317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:44:06.363317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:44:06.363317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:44:06.363317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:44:06.363317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:44:06.363317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:44:06.363317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:44:06.363317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:44:06.363317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:44:06.363317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:44:06.363317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:44:06.363317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:44:06.363317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:44:06.363317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:44:06.364589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:44:06.364589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:44:06.364589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:44:06.364589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:44:06.364589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:44:06.364589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:44:06.364589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:44:06.364589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:44:06.364589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:44:06.364589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:44:06.364589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:44:06.364589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:44:06.364589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:44:06.364589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:44:06.364589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:44:06.364589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:44:06.365726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:44:06.365726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:44:06.365726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:44:06.365726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:44:06.365726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:44:06.365726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:44:06.365726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:44:06.365726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:44:06.365726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:44:06.365726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:44:06.365726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:44:06.365726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:44:06.365726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 578/1141] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:44:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:44:06.371379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c4698-1b63-4e34-8d91-4e17f5a0bf6e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:06.432509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:06.471639 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c4698-1b63-4e34-8d91-4e17f5a0bf6e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:06.505862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3a7d4-2516-4a32-9f25-97993924acaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:06.506460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3a7d4-2516-4a32-9f25-97993924acaf None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:06.506739 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3a7d4-2516-4a32-9f25-97993924acaf None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:06.507004 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3a7d4-2516-4a32-9f25-97993924acaf None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:06.507656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3a7d4-2516-4a32-9f25-97993924acaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:44:06.507978 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3a7d4-2516-4a32-9f25-97993924acaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:06.509353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3a7d4-2516-4a32-9f25-97993924acaf None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:06.517113 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 564/1142] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:44:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:06.523114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e693d23-4a93-44dd-bd49-06f3e6c8ed76 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:06.625376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e693d23-4a93-44dd-bd49-06f3e6c8ed76 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:06.659616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11dfaa8-50f6-4cc6-96d1-7b3a70af65fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:06.660272 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11dfaa8-50f6-4cc6-96d1-7b3a70af65fe None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:06.660555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11dfaa8-50f6-4cc6-96d1-7b3a70af65fe None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:06.660833 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11dfaa8-50f6-4cc6-96d1-7b3a70af65fe None admin] PATH_INFO: `/v3/users/b5d86d0d369b44f6885f6dcba0fde4de` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:06.674777 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11dfaa8-50f6-4cc6-96d1-7b3a70af65fe None admin] RBAC: Authorizing `identity:get_user(user_id=b5d86d0d369b44f6885f6dcba0fde4de)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:06.676247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11dfaa8-50f6-4cc6-96d1-7b3a70af65fe None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:06.677899 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 579/1143] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:44:06 2022] GET /identity/v3/users/b5d86d0d369b44f6885f6dcba0fde4de => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:06.684475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3a7d4-2516-4a32-9f25-97993924acaf None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:06.785580 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3a7d4-2516-4a32-9f25-97993924acaf None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:06.811703 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:06.823888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276638b-5f8c-41ce-ae12-f30fb30d0cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:06.824543 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276638b-5f8c-41ce-ae12-f30fb30d0cec None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:06.824812 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276638b-5f8c-41ce-ae12-f30fb30d0cec None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:06.825079 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276638b-5f8c-41ce-ae12-f30fb30d0cec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:06.894026 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e276638b-5f8c-41ce-ae12-f30fb30d0cec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:06.895462 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e276638b-5f8c-41ce-ae12-f30fb30d0cec None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:06.995424 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 565/1144] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:44:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:44:07.001438 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11dfaa8-50f6-4cc6-96d1-7b3a70af65fe None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:07.030953 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:07.112493 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11dfaa8-50f6-4cc6-96d1-7b3a70af65fe None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:07.149194 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83cace-9e50-4041-a76b-2fe4c3852e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:07.149843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83cace-9e50-4041-a76b-2fe4c3852e1e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:07.150119 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83cace-9e50-4041-a76b-2fe4c3852e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:07.150393 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83cace-9e50-4041-a76b-2fe4c3852e1e None admin] PATH_INFO: `/v3/projects/b5430204bd1042dba363bcabfa0bbad4` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:07.151525 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83cace-9e50-4041-a76b-2fe4c3852e1e None admin] RBAC: Authorizing `identity:get_project(project_id=b5430204bd1042dba363bcabfa0bbad4)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:07.152988 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83cace-9e50-4041-a76b-2fe4c3852e1e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:07.154641 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 580/1145] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:44:07 2022] GET /identity/v3/projects/b5430204bd1042dba363bcabfa0bbad4 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:07.160882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276638b-5f8c-41ce-ae12-f30fb30d0cec None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:07.186346 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:07.260836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276638b-5f8c-41ce-ae12-f30fb30d0cec None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:07.294017 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c163e6e-6541-4c9d-adf9-6081bda41862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:07.294607 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c163e6e-6541-4c9d-adf9-6081bda41862 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:07.294882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c163e6e-6541-4c9d-adf9-6081bda41862 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:07.295148 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c163e6e-6541-4c9d-adf9-6081bda41862 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:07.295859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c163e6e-6541-4c9d-adf9-6081bda41862 None admin] RBAC: Adding query filter params (role.id=b13ffa72344c4856901b1ba003e273fd, scope.project.id=b5430204bd1042dba363bcabfa0bbad4, user.id=b5d86d0d369b44f6885f6dcba0fde4de) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:44:07.296186 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c163e6e-6541-4c9d-adf9-6081bda41862 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:07.297590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c163e6e-6541-4c9d-adf9-6081bda41862 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:07.302778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 566/1146] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:44:07 2022] GET /identity/v3/role_assignments?user.id=b5d86d0d369b44f6885f6dcba0fde4de&scope.project.id=b5430204bd1042dba363bcabfa0bbad4&role.id=b13ffa72344c4856901b1ba003e273fd => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:09.197664 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e4f04-48f1-48ca-af18-330452642d32 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:09.198230 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e4f04-48f1-48ca-af18-330452642d32 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:09.198230 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e4f04-48f1-48ca-af18-330452642d32 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:09.199596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 581/1147] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:44:09.204916 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436f238-67ea-4218-87b1-a9be8b923056 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:09.205327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436f238-67ea-4218-87b1-a9be8b923056 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:09.205454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436f238-67ea-4218-87b1-a9be8b923056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:09.224020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-3436f238-67ea-4218-87b1-a9be8b923056 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:44:09.225307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3436f238-67ea-4218-87b1-a9be8b923056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:44:09.292845 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 567/1148] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:44:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:44:09.563694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e4f04-48f1-48ca-af18-330452642d32 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:09.673956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:09.690463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e4f04-48f1-48ca-af18-330452642d32 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:09.746434 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aaa3ff-f124-4e3a-8d56-6052fc734a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:09.747064 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaa3ff-f124-4e3a-8d56-6052fc734a82 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:09.747344 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaa3ff-f124-4e3a-8d56-6052fc734a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:09.747614 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaa3ff-f124-4e3a-8d56-6052fc734a82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:09.758550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aaa3ff-f124-4e3a-8d56-6052fc734a82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:44:09.763289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-83aaa3ff-f124-4e3a-8d56-6052fc734a82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:44:09.763289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:44:09.763289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:44:09.763289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:44:09.763289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:44:09.763289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:44:09.763289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:44:09.763289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:44:09.763289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:44:09.763289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:44:09.763289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:44:09.763289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:44:09.763289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:44:09.763289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:44:09.763289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:44:09.763289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:44:09.764280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:44:09.764280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:44:09.764280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:44:09.764280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:44:09.764280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:44:09.764280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:44:09.764280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:44:09.764280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:44:09.764280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:44:09.764280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:44:09.764280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:44:09.764280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:44:09.764280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:44:09.764280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:44:09.764280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:44:09.764280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:44:09.765166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:44:09.765166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:44:09.765166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:44:09.765166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:44:09.765166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:44:09.765166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:44:09.765166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:44:09.765166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:44:09.765166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:44:09.765166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:44:09.765166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:44:09.765166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:44:09.765166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 582/1149] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:44:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:44:09.770485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436f238-67ea-4218-87b1-a9be8b923056 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:09.788456 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:09.871672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436f238-67ea-4218-87b1-a9be8b923056 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:09.906865 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220b467-9014-4f66-ad04-f218a2a412bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:09.907482 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220b467-9014-4f66-ad04-f218a2a412bd None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:09.907760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220b467-9014-4f66-ad04-f218a2a412bd None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:09.908032 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220b467-9014-4f66-ad04-f218a2a412bd None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:09.908735 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a220b467-9014-4f66-ad04-f218a2a412bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:44:09.909036 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a220b467-9014-4f66-ad04-f218a2a412bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:09.910404 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a220b467-9014-4f66-ad04-f218a2a412bd None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:09.919043 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 568/1150] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:44:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:09.924976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aaa3ff-f124-4e3a-8d56-6052fc734a82 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:10.025747 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aaa3ff-f124-4e3a-8d56-6052fc734a82 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:10.061278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc071fc-6091-4e4c-80df-907db5aa4450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:10.061900 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc071fc-6091-4e4c-80df-907db5aa4450 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:10.062182 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc071fc-6091-4e4c-80df-907db5aa4450 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:10.062441 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc071fc-6091-4e4c-80df-907db5aa4450 None admin] PATH_INFO: `/v3/users/b5d86d0d369b44f6885f6dcba0fde4de` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:10.063549 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc071fc-6091-4e4c-80df-907db5aa4450 None admin] RBAC: Authorizing `identity:get_user(user_id=b5d86d0d369b44f6885f6dcba0fde4de)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:10.064999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc071fc-6091-4e4c-80df-907db5aa4450 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:10.066568 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 583/1151] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:44:09 2022] GET /identity/v3/users/b5d86d0d369b44f6885f6dcba0fde4de => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:10.072940 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220b467-9014-4f66-ad04-f218a2a412bd None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:10.157780 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:10.178522 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220b467-9014-4f66-ad04-f218a2a412bd None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:10.214957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb23b79-552f-4328-b394-11503647b21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:10.215646 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb23b79-552f-4328-b394-11503647b21e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:10.215837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb23b79-552f-4328-b394-11503647b21e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:10.216123 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb23b79-552f-4328-b394-11503647b21e None admin] PATH_INFO: `/v3/projects/b5430204bd1042dba363bcabfa0bbad4` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:10.217448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb23b79-552f-4328-b394-11503647b21e None admin] RBAC: Authorizing `identity:get_project(project_id=b5430204bd1042dba363bcabfa0bbad4)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:10.218861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb23b79-552f-4328-b394-11503647b21e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:10.220537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 569/1152] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:44:10 2022] GET /identity/v3/projects/b5430204bd1042dba363bcabfa0bbad4 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:10.226645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc071fc-6091-4e4c-80df-907db5aa4450 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:10.249247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:10.329671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc071fc-6091-4e4c-80df-907db5aa4450 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:10.364220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ff27c-6915-47d5-af7b-2d29006d4d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:10.364884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ff27c-6915-47d5-af7b-2d29006d4d6c None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:10.365163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ff27c-6915-47d5-af7b-2d29006d4d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:10.365435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ff27c-6915-47d5-af7b-2d29006d4d6c None admin] PATH_INFO: `/v3/projects/b5430204bd1042dba363bcabfa0bbad4/users/b5d86d0d369b44f6885f6dcba0fde4de/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:10.367915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ff27c-6915-47d5-af7b-2d29006d4d6c None admin] RBAC: Authorizing `identity:create_grant(project_id=b5430204bd1042dba363bcabfa0bbad4, user_id=b5d86d0d369b44f6885f6dcba0fde4de, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:10.369593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ff27c-6915-47d5-af7b-2d29006d4d6c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:10.387974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 584/1153] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:44:10 2022] PUT /identity/v3/projects/b5430204bd1042dba363bcabfa0bbad4/users/b5d86d0d369b44f6885f6dcba0fde4de/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:44:12.328831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf103c9-3597-40b4-8086-599871a84e39 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:12.329414 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf103c9-3597-40b4-8086-599871a84e39 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:12.329414 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf103c9-3597-40b4-8086-599871a84e39 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:12.330715 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 570/1154] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:44:12.335939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139edc7c-bd1e-446d-bad6-ecb7810209ef None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:12.336627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139edc7c-bd1e-446d-bad6-ecb7810209ef None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:12.336627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139edc7c-bd1e-446d-bad6-ecb7810209ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:12.354966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-139edc7c-bd1e-446d-bad6-ecb7810209ef None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:44:12.356184 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-139edc7c-bd1e-446d-bad6-ecb7810209ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:44:12.424572 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 585/1155] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:44:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:44:12.696418 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf103c9-3597-40b4-8086-599871a84e39 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:12.798442 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf103c9-3597-40b4-8086-599871a84e39 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:12.833365 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da855e36-3d5a-42d0-853e-f5701044b94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:12.833995 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da855e36-3d5a-42d0-853e-f5701044b94e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:12.834277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da855e36-3d5a-42d0-853e-f5701044b94e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:12.834550 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da855e36-3d5a-42d0-853e-f5701044b94e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:12.846602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da855e36-3d5a-42d0-853e-f5701044b94e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:44:12.851356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-da855e36-3d5a-42d0-853e-f5701044b94e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:44:12.851356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:44:12.851356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:44:12.851356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:44:12.851356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:44:12.851356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:44:12.851356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:44:12.851356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:44:12.851356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:44:12.851356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:44:12.851356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:44:12.851356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:44:12.851356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:44:12.851356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:44:12.851356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:44:12.851356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:44:12.853255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:44:12.853255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:44:12.853255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:44:12.853255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:44:12.853255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:44:12.853255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:44:12.853255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:44:12.853255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:44:12.853255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:44:12.853255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:44:12.853255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:44:12.853255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:44:12.853255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:44:12.853255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:44:12.853255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:44:12.853255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:44:12.855434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:44:12.855434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:44:12.855434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:44:12.855434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:44:12.855434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:44:12.855434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:44:12.855434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:44:12.855434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:44:12.855434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:44:12.855434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:44:12.855434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:44:12.855434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:44:12.855434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 571/1156] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:44:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:44:12.859410 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139edc7c-bd1e-446d-bad6-ecb7810209ef None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:12.871236 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:12.961900 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139edc7c-bd1e-446d-bad6-ecb7810209ef None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:12.995929 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa3530-1afd-418a-9df1-e3b5f07a0a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:12.996561 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa3530-1afd-418a-9df1-e3b5f07a0a56 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:12.996841 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa3530-1afd-418a-9df1-e3b5f07a0a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:12.997111 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa3530-1afd-418a-9df1-e3b5f07a0a56 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:12.997776 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa3530-1afd-418a-9df1-e3b5f07a0a56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:44:12.998080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa3530-1afd-418a-9df1-e3b5f07a0a56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:12.999442 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa3530-1afd-418a-9df1-e3b5f07a0a56 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:13.007629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 586/1157] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:44:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:13.014132 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da855e36-3d5a-42d0-853e-f5701044b94e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:13.026230 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:13.116382 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da855e36-3d5a-42d0-853e-f5701044b94e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:13.152372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3aced-00bf-4d1e-b4d2-730e5dc8cb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:13.153002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3aced-00bf-4d1e-b4d2-730e5dc8cb89 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:13.153284 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3aced-00bf-4d1e-b4d2-730e5dc8cb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:13.153559 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3aced-00bf-4d1e-b4d2-730e5dc8cb89 None admin] PATH_INFO: `/v3/users/b5d86d0d369b44f6885f6dcba0fde4de` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:13.154701 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c3aced-00bf-4d1e-b4d2-730e5dc8cb89 None admin] RBAC: Authorizing `identity:get_user(user_id=b5d86d0d369b44f6885f6dcba0fde4de)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:13.156157 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c3aced-00bf-4d1e-b4d2-730e5dc8cb89 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:13.157757 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 572/1158] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:44:13 2022] GET /identity/v3/users/b5d86d0d369b44f6885f6dcba0fde4de => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:13.163776 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa3530-1afd-418a-9df1-e3b5f07a0a56 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:13.233954 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:13.266073 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa3530-1afd-418a-9df1-e3b5f07a0a56 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:13.300436 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723916a-e490-4632-8501-8c7c8ad83077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:13.301047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723916a-e490-4632-8501-8c7c8ad83077 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:13.301327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723916a-e490-4632-8501-8c7c8ad83077 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:13.301597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723916a-e490-4632-8501-8c7c8ad83077 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:13.369887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723916a-e490-4632-8501-8c7c8ad83077 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:13.375281 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723916a-e490-4632-8501-8c7c8ad83077 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:13.450205 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:13.481615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 587/1159] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:44:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:44:13.487618 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3aced-00bf-4d1e-b4d2-730e5dc8cb89 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:13.569381 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:13.593953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3aced-00bf-4d1e-b4d2-730e5dc8cb89 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:13.628334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc97c83-9ffd-4745-90e8-d882c921a1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:13.628969 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc97c83-9ffd-4745-90e8-d882c921a1c6 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:13.629249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc97c83-9ffd-4745-90e8-d882c921a1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:13.629521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc97c83-9ffd-4745-90e8-d882c921a1c6 None admin] PATH_INFO: `/v3/projects/b5430204bd1042dba363bcabfa0bbad4` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:13.630639 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc97c83-9ffd-4745-90e8-d882c921a1c6 None admin] RBAC: Authorizing `identity:get_project(project_id=b5430204bd1042dba363bcabfa0bbad4)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:13.632123 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc97c83-9ffd-4745-90e8-d882c921a1c6 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:13.633989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 573/1160] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:44:13 2022] GET /identity/v3/projects/b5430204bd1042dba363bcabfa0bbad4 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:13.640171 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723916a-e490-4632-8501-8c7c8ad83077 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:13.739093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723916a-e490-4632-8501-8c7c8ad83077 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:13.772426 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25839b3b-ee25-4097-a549-5312a5598a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:13.773019 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25839b3b-ee25-4097-a549-5312a5598a01 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:13.773295 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25839b3b-ee25-4097-a549-5312a5598a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:13.773561 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25839b3b-ee25-4097-a549-5312a5598a01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:13.774339 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25839b3b-ee25-4097-a549-5312a5598a01 None admin] RBAC: Adding query filter params (role.id=b13ffa72344c4856901b1ba003e273fd, scope.project.id=b5430204bd1042dba363bcabfa0bbad4, user.id=b5d86d0d369b44f6885f6dcba0fde4de) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:44:13.774645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25839b3b-ee25-4097-a549-5312a5598a01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:13.776061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25839b3b-ee25-4097-a549-5312a5598a01 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:13.781683 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 588/1161] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:44:13 2022] GET /identity/v3/role_assignments?user.id=b5d86d0d369b44f6885f6dcba0fde4de&scope.project.id=b5430204bd1042dba363bcabfa0bbad4&role.id=b13ffa72344c4856901b1ba003e273fd => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:15.743246 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c066149-9120-4823-8b80-d66d1c330445 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:15.743717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c066149-9120-4823-8b80-d66d1c330445 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:15.743809 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c066149-9120-4823-8b80-d66d1c330445 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:15.745254 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 574/1162] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:44:15.751056 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b7f10-0cc1-4003-aedf-e7acdb07b41b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:15.751511 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b7f10-0cc1-4003-aedf-e7acdb07b41b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:15.751628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b7f10-0cc1-4003-aedf-e7acdb07b41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:15.769589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e12b7f10-0cc1-4003-aedf-e7acdb07b41b None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:44:15.770766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e12b7f10-0cc1-4003-aedf-e7acdb07b41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:44:15.800213 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:15.839211 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 589/1163] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:44:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:44:16.108302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c066149-9120-4823-8b80-d66d1c330445 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:16.210381 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c066149-9120-4823-8b80-d66d1c330445 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:16.245158 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa3841-bfd1-4b46-baec-cc2cad295836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:16.245758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa3841-bfd1-4b46-baec-cc2cad295836 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:16.246036 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa3841-bfd1-4b46-baec-cc2cad295836 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:16.246303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa3841-bfd1-4b46-baec-cc2cad295836 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:16.262509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:16.315847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa3841-bfd1-4b46-baec-cc2cad295836 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:16.317317 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa3841-bfd1-4b46-baec-cc2cad295836 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:16.435790 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 575/1164] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:44:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:44:16.441841 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b7f10-0cc1-4003-aedf-e7acdb07b41b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:16.545148 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b7f10-0cc1-4003-aedf-e7acdb07b41b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:16.579482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd3801-ac85-4a78-a613-de63c30bd34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:16.580139 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd3801-ac85-4a78-a613-de63c30bd34b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:16.580422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd3801-ac85-4a78-a613-de63c30bd34b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:16.580692 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd3801-ac85-4a78-a613-de63c30bd34b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:16.588336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd3801-ac85-4a78-a613-de63c30bd34b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:16.590043 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd3801-ac85-4a78-a613-de63c30bd34b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:16.603682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d0bd3801-ac85-4a78-a613-de63c30bd34b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 08:44:16.603682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:44:16.603682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 08:44:16.603682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 08:44:16.603682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:44:16.603682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:44:16.603682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:44:16.603682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:44:16.603682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 08:44:16.603682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:44:16.603682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:44:16.603682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:44:16.603682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:44:16.603682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:44:16.603682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:44:16.603682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:44:16.603682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:44:16.605046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:44:16.605046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:44:16.605046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:44:16.605046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:44:16.605046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:44:16.605046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:44:16.605046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:44:16.605046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:44:16.605046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:44:16.605046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:44:16.605046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:44:16.605046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:44:16.605046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:44:16.605046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:44:16.605046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:44:16.605046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:44:16.606300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:44:16.606300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:44:16.606300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:44:16.606300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:44:16.606300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:44:16.606300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:44:16.606300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:44:16.606300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:44:16.606300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:44:16.606300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:44:16.606300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 08:44:16.606300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:44:16.606300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 08:44:16.606300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:44:16.606300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 590/1165] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 24 08:44:16 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:44:16.611426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa3841-bfd1-4b46-baec-cc2cad295836 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:16.661071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:16.718045 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa3841-bfd1-4b46-baec-cc2cad295836 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:16.753508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bb3d7-eb03-4562-a870-c2e5bfd44c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:16.754116 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bb3d7-eb03-4562-a870-c2e5bfd44c1e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:16.754392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bb3d7-eb03-4562-a870-c2e5bfd44c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:16.754659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bb3d7-eb03-4562-a870-c2e5bfd44c1e None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:16.755293 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bb3d7-eb03-4562-a870-c2e5bfd44c1e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:44:16.755591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bb3d7-eb03-4562-a870-c2e5bfd44c1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:16.756984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bb3d7-eb03-4562-a870-c2e5bfd44c1e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:16.764815 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 576/1166] 10.222.0.67 () {56 vars in 1115 bytes} [Wed Aug 24 08:44:16 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:18.674040 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2722d9-3898-4b1c-9021-7480306ccdc0 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:18.674573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2722d9-3898-4b1c-9021-7480306ccdc0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:18.674573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2722d9-3898-4b1c-9021-7480306ccdc0 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:18.675980 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 591/1167] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:44:18.682250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7279de-b86b-45bb-a477-62bf49ee4b53 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:18.682726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7279de-b86b-45bb-a477-62bf49ee4b53 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:18.682822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7279de-b86b-45bb-a477-62bf49ee4b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:18.701116 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-2b7279de-b86b-45bb-a477-62bf49ee4b53 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:44:18.702295 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2b7279de-b86b-45bb-a477-62bf49ee4b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:44:18.771096 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 577/1168] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:44:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:44:19.040892 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2722d9-3898-4b1c-9021-7480306ccdc0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:19.072484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:19.150060 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2722d9-3898-4b1c-9021-7480306ccdc0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:19.184203 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca20a5a-4950-42ed-b83e-3ecbd7ab3794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:19.184804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca20a5a-4950-42ed-b83e-3ecbd7ab3794 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:19.185078 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca20a5a-4950-42ed-b83e-3ecbd7ab3794 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:19.185343 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca20a5a-4950-42ed-b83e-3ecbd7ab3794 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:19.185995 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca20a5a-4950-42ed-b83e-3ecbd7ab3794 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:19.187344 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca20a5a-4950-42ed-b83e-3ecbd7ab3794 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:19.203290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 592/1169] 10.222.0.67 () {60 vars in 1138 bytes} [Wed Aug 24 08:44:19 2022] POST /identity/v3/domains => generated 267 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:44:21.152563 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47197d-10ba-42c7-81ea-403142d6f6a1 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:21.153132 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47197d-10ba-42c7-81ea-403142d6f6a1 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:21.153132 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47197d-10ba-42c7-81ea-403142d6f6a1 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:21.154455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 578/1170] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:44:21.159738 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5cdb6b-ef75-4b0a-b5c7-069606275891 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:21.160220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5cdb6b-ef75-4b0a-b5c7-069606275891 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:21.160338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5cdb6b-ef75-4b0a-b5c7-069606275891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:21.177983 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0d5cdb6b-ef75-4b0a-b5c7-069606275891 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:44:21.179161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0d5cdb6b-ef75-4b0a-b5c7-069606275891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:44:21.247052 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 593/1171] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:44:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:44:21.519040 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47197d-10ba-42c7-81ea-403142d6f6a1 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:21.561208 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:21.623347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47197d-10ba-42c7-81ea-403142d6f6a1 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:21.657282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3ed5b-a540-4414-8f94-96af3184002b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:21.657912 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3ed5b-a540-4414-8f94-96af3184002b None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:21.658189 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3ed5b-a540-4414-8f94-96af3184002b None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:21.658459 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3ed5b-a540-4414-8f94-96af3184002b None admin] PATH_INFO: `/v3/domains/1f99c43b9330454aa1b632c531089a48` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:21.669534 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb3ed5b-a540-4414-8f94-96af3184002b None admin] RBAC: Authorizing `identity:get_domain(domain_id=1f99c43b9330454aa1b632c531089a48)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:21.670969 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb3ed5b-a540-4414-8f94-96af3184002b None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:21.672648 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 579/1172] 10.222.0.67 () {56 vars in 1183 bytes} [Wed Aug 24 08:44:21 2022] GET /identity/v3/domains/1f99c43b9330454aa1b632c531089a48 => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:21.678561 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5cdb6b-ef75-4b0a-b5c7-069606275891 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:21.697828 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:21.783215 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5cdb6b-ef75-4b0a-b5c7-069606275891 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:21.817130 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1130bea-800b-4e40-9c8c-8c0c69bea4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:21.817725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1130bea-800b-4e40-9c8c-8c0c69bea4fa None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:21.818003 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1130bea-800b-4e40-9c8c-8c0c69bea4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:21.818274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1130bea-800b-4e40-9c8c-8c0c69bea4fa None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:21.818930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1130bea-800b-4e40-9c8c-8c0c69bea4fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:21.820505 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1130bea-800b-4e40-9c8c-8c0c69bea4fa None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:21.849378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 594/1173] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 24 08:44:21 2022] POST /identity/v3/projects => generated 361 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:44:23.849996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf977c-31f6-4658-ba40-24f7c9941e6a None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:23.850590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf977c-31f6-4658-ba40-24f7c9941e6a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:23.850590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf977c-31f6-4658-ba40-24f7c9941e6a None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:23.851898 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 580/1174] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:44:23.858662 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175c0bc-3b6c-422a-acd3-bd35360d5879 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:23.859121 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175c0bc-3b6c-422a-acd3-bd35360d5879 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:23.859222 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175c0bc-3b6c-422a-acd3-bd35360d5879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:23.879485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6175c0bc-3b6c-422a-acd3-bd35360d5879 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:44:23.880717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6175c0bc-3b6c-422a-acd3-bd35360d5879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:44:23.946269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:23.953008 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 595/1175] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:44:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:44:24.228170 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf977c-31f6-4658-ba40-24f7c9941e6a None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:24.325539 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:24.330762 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf977c-31f6-4658-ba40-24f7c9941e6a None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:24.366897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a45bc-30c1-4fbc-af26-279b3cf88097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:24.367543 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a45bc-30c1-4fbc-af26-279b3cf88097 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:24.367822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a45bc-30c1-4fbc-af26-279b3cf88097 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:24.368275 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a45bc-30c1-4fbc-af26-279b3cf88097 None admin] PATH_INFO: `/v3/domains/1f99c43b9330454aa1b632c531089a48` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:24.369272 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3a45bc-30c1-4fbc-af26-279b3cf88097 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1f99c43b9330454aa1b632c531089a48)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:24.370704 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3a45bc-30c1-4fbc-af26-279b3cf88097 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:24.372334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 581/1176] 10.222.0.67 () {56 vars in 1183 bytes} [Wed Aug 24 08:44:24 2022] GET /identity/v3/domains/1f99c43b9330454aa1b632c531089a48 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:24.378381 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175c0bc-3b6c-422a-acd3-bd35360d5879 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:24.484602 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175c0bc-3b6c-422a-acd3-bd35360d5879 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:24.520617 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3090e69-f2b8-4c0c-953d-cb5705c3c4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:24.521227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3090e69-f2b8-4c0c-953d-cb5705c3c4c1 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:24.521505 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3090e69-f2b8-4c0c-953d-cb5705c3c4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:24.521765 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3090e69-f2b8-4c0c-953d-cb5705c3c4c1 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:24.522426 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3090e69-f2b8-4c0c-953d-cb5705c3c4c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:24.524608 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3090e69-f2b8-4c0c-953d-cb5705c3c4c1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:24.548197 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 596/1177] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 24 08:44:24 2022] POST /identity/v3/users => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:44:26.569518 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5342c-451a-4f71-87a2-cea44ec0d79e None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:26.570096 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5342c-451a-4f71-87a2-cea44ec0d79e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:26.570096 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5342c-451a-4f71-87a2-cea44ec0d79e None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:26.572224 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 582/1178] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:44:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:44:26.577615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0422c4-1e2c-4841-81b2-b7ef4c0b8414 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:26.578050 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0422c4-1e2c-4841-81b2-b7ef4c0b8414 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:26.578142 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0422c4-1e2c-4841-81b2-b7ef4c0b8414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:26.597770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-be0422c4-1e2c-4841-81b2-b7ef4c0b8414 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:44:26.598936 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-be0422c4-1e2c-4841-81b2-b7ef4c0b8414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:44:26.647660 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:26.666339 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 597/1179] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:44:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 08:44:26.940825 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5342c-451a-4f71-87a2-cea44ec0d79e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:27.041478 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5342c-451a-4f71-87a2-cea44ec0d79e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:27.076644 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a385578-c56c-4e54-9f9a-41507db1f4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:27.077275 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a385578-c56c-4e54-9f9a-41507db1f4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:27.077556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a385578-c56c-4e54-9f9a-41507db1f4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:27.077829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a385578-c56c-4e54-9f9a-41507db1f4a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:27.088317 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a385578-c56c-4e54-9f9a-41507db1f4a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:44:27.092931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-2a385578-c56c-4e54-9f9a-41507db1f4a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:44:27.092931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:44:27.092931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:44:27.092931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:44:27.092931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:44:27.092931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:44:27.092931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:44:27.092931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:44:27.092931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:44:27.092931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:44:27.092931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:44:27.092931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:44:27.092931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:44:27.092931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:44:27.092931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:44:27.092931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:44:27.094048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:44:27.094048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:44:27.094048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:44:27.094048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:44:27.094048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:44:27.094048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:44:27.094048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:44:27.094048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:44:27.094048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:44:27.094048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:44:27.094048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:44:27.094048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:44:27.094048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:44:27.094048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:44:27.094048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:44:27.094048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:44:27.095108 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:44:27.095108 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:44:27.095108 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:44:27.095108 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:44:27.095108 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:44:27.095108 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:44:27.095108 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:44:27.095108 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:44:27.095108 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:44:27.095108 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:44:27.095108 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:44:27.095108 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:44:27.095108 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 583/1180] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:44:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:44:27.100139 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0422c4-1e2c-4841-81b2-b7ef4c0b8414 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:27.199159 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0422c4-1e2c-4841-81b2-b7ef4c0b8414 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:27.235277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9562d92-7f51-4f93-ad5c-5cd70bf32a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:27.235885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562d92-7f51-4f93-ad5c-5cd70bf32a6b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:27.236189 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562d92-7f51-4f93-ad5c-5cd70bf32a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:27.236461 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562d92-7f51-4f93-ad5c-5cd70bf32a6b None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:27.237125 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562d92-7f51-4f93-ad5c-5cd70bf32a6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:44:27.237425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562d92-7f51-4f93-ad5c-5cd70bf32a6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:27.238782 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562d92-7f51-4f93-ad5c-5cd70bf32a6b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:27.246952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 598/1181] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:44:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:27.252859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a385578-c56c-4e54-9f9a-41507db1f4a4 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:27.281652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:27.353725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a385578-c56c-4e54-9f9a-41507db1f4a4 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:27.387751 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b3c92-7e7b-48cc-83cb-ac08e89812e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:27.388410 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b3c92-7e7b-48cc-83cb-ac08e89812e4 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:27.388688 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b3c92-7e7b-48cc-83cb-ac08e89812e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:27.388957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b3c92-7e7b-48cc-83cb-ac08e89812e4 None admin] PATH_INFO: `/v3/users/16b73fff7a0b4214b1803a7396b89f6c` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:27.402841 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b3c92-7e7b-48cc-83cb-ac08e89812e4 None admin] RBAC: Authorizing `identity:get_user(user_id=16b73fff7a0b4214b1803a7396b89f6c)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:27.404313 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b3c92-7e7b-48cc-83cb-ac08e89812e4 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:27.405949 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 584/1182] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:44:27 2022] GET /identity/v3/users/16b73fff7a0b4214b1803a7396b89f6c => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:27.411883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9562d92-7f51-4f93-ad5c-5cd70bf32a6b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:27.458871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:27.513233 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9562d92-7f51-4f93-ad5c-5cd70bf32a6b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:27.546012 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3333f92-9c18-491b-b638-69a92bef5dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:27.546608 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3333f92-9c18-491b-b638-69a92bef5dbe None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:27.546885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3333f92-9c18-491b-b638-69a92bef5dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:27.547156 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3333f92-9c18-491b-b638-69a92bef5dbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:27.613183 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3333f92-9c18-491b-b638-69a92bef5dbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:27.614610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3333f92-9c18-491b-b638-69a92bef5dbe None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:27.665642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:27.721655 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 599/1183] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:44:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:44:27.727450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b3c92-7e7b-48cc-83cb-ac08e89812e4 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:27.814001 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:27.830377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b3c92-7e7b-48cc-83cb-ac08e89812e4 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:27.863306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364292c-7014-4e3b-a60a-c876af467074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:27.863936 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364292c-7014-4e3b-a60a-c876af467074 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:27.864250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364292c-7014-4e3b-a60a-c876af467074 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:27.864524 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364292c-7014-4e3b-a60a-c876af467074 None admin] PATH_INFO: `/v3/projects/d66fc13c1f064f1990d2a1ebe836ffac` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:27.865643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364292c-7014-4e3b-a60a-c876af467074 None admin] RBAC: Authorizing `identity:get_project(project_id=d66fc13c1f064f1990d2a1ebe836ffac)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:27.867058 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364292c-7014-4e3b-a60a-c876af467074 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:27.869271 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 585/1184] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:44:27 2022] GET /identity/v3/projects/d66fc13c1f064f1990d2a1ebe836ffac => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:27.875476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3333f92-9c18-491b-b638-69a92bef5dbe None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:27.975499 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3333f92-9c18-491b-b638-69a92bef5dbe None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:28.009145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe755b5-f2b3-4287-b615-dc34c529e4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:28.009734 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe755b5-f2b3-4287-b615-dc34c529e4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:28.010008 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe755b5-f2b3-4287-b615-dc34c529e4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:28.010273 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe755b5-f2b3-4287-b615-dc34c529e4e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:28.010985 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe755b5-f2b3-4287-b615-dc34c529e4e0 None admin] RBAC: Adding query filter params (role.id=b13ffa72344c4856901b1ba003e273fd, scope.project.id=d66fc13c1f064f1990d2a1ebe836ffac, user.id=16b73fff7a0b4214b1803a7396b89f6c) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:44:28.011285 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe755b5-f2b3-4287-b615-dc34c529e4e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:28.012733 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe755b5-f2b3-4287-b615-dc34c529e4e0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:28.017946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 600/1185] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:44:27 2022] GET /identity/v3/role_assignments?user.id=16b73fff7a0b4214b1803a7396b89f6c&scope.project.id=d66fc13c1f064f1990d2a1ebe836ffac&role.id=b13ffa72344c4856901b1ba003e273fd => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:29.936358 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403bebd-23a9-4054-9860-abb4265bc319 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:29.936919 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403bebd-23a9-4054-9860-abb4265bc319 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:29.936919 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403bebd-23a9-4054-9860-abb4265bc319 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:29.938284 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 586/1186] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:44:29.943521 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecc168-5539-47ed-878c-a2f1188e9e3c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:29.943936 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecc168-5539-47ed-878c-a2f1188e9e3c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:29.944057 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecc168-5539-47ed-878c-a2f1188e9e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:29.962685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-02ecc168-5539-47ed-878c-a2f1188e9e3c None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:44:29.963856 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-02ecc168-5539-47ed-878c-a2f1188e9e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:44:29.966662 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:30.031133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 601/1187] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:44:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:44:30.307504 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403bebd-23a9-4054-9860-abb4265bc319 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:30.410693 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403bebd-23a9-4054-9860-abb4265bc319 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:30.444741 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37603e97-e6e6-476f-ac8b-52a74d47c14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:30.445375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37603e97-e6e6-476f-ac8b-52a74d47c14e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:30.445655 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37603e97-e6e6-476f-ac8b-52a74d47c14e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:30.445929 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37603e97-e6e6-476f-ac8b-52a74d47c14e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:30.457337 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37603e97-e6e6-476f-ac8b-52a74d47c14e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:44:30.462746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-37603e97-e6e6-476f-ac8b-52a74d47c14e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:44:30.462746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:44:30.462746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:44:30.462746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:44:30.462746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:44:30.462746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:44:30.462746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:44:30.462746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:44:30.462746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:44:30.462746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:44:30.462746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:44:30.462746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:44:30.462746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:44:30.462746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:44:30.462746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:44:30.462746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:44:30.464163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:44:30.464163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:44:30.464163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:44:30.464163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:44:30.464163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:44:30.464163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:44:30.464163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:44:30.464163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:44:30.464163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:44:30.464163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:44:30.464163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:44:30.464163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:44:30.464163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:44:30.464163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:44:30.464163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:44:30.464163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:44:30.465101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:44:30.465101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:44:30.465101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:44:30.465101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:44:30.465101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:44:30.465101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:44:30.465101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:44:30.465101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:44:30.465101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:44:30.465101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:44:30.465101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:44:30.465101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:44:30.465101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 587/1188] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:44:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:44:30.470049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecc168-5539-47ed-878c-a2f1188e9e3c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:30.575431 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecc168-5539-47ed-878c-a2f1188e9e3c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:30.609370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bf591-d7a1-4fda-ae8a-9c093c60a3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:30.609984 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bf591-d7a1-4fda-ae8a-9c093c60a3b8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:30.610265 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bf591-d7a1-4fda-ae8a-9c093c60a3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:30.610535 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bf591-d7a1-4fda-ae8a-9c093c60a3b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:30.611204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23bf591-d7a1-4fda-ae8a-9c093c60a3b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:44:30.611510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23bf591-d7a1-4fda-ae8a-9c093c60a3b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:30.612906 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23bf591-d7a1-4fda-ae8a-9c093c60a3b8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:30.622005 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 602/1189] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:44:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:30.627921 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37603e97-e6e6-476f-ac8b-52a74d47c14e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:30.644229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:30.728607 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37603e97-e6e6-476f-ac8b-52a74d47c14e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:30.767840 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983cbd8-66a4-4b2c-b1bb-6673e12939d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:30.768505 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983cbd8-66a4-4b2c-b1bb-6673e12939d0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:30.768783 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983cbd8-66a4-4b2c-b1bb-6673e12939d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:30.769055 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983cbd8-66a4-4b2c-b1bb-6673e12939d0 None admin] PATH_INFO: `/v3/users/16b73fff7a0b4214b1803a7396b89f6c` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:30.770186 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983cbd8-66a4-4b2c-b1bb-6673e12939d0 None admin] RBAC: Authorizing `identity:get_user(user_id=16b73fff7a0b4214b1803a7396b89f6c)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:30.771624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983cbd8-66a4-4b2c-b1bb-6673e12939d0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:30.773262 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 588/1190] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:44:30 2022] GET /identity/v3/users/16b73fff7a0b4214b1803a7396b89f6c => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:30.779235 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bf591-d7a1-4fda-ae8a-9c093c60a3b8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:30.782108 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:30.878881 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bf591-d7a1-4fda-ae8a-9c093c60a3b8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:30.912379 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0186f93-e56f-46ba-8fca-101726cf238a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:30.913009 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0186f93-e56f-46ba-8fca-101726cf238a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:30.913279 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0186f93-e56f-46ba-8fca-101726cf238a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:30.913546 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0186f93-e56f-46ba-8fca-101726cf238a None admin] PATH_INFO: `/v3/projects/d66fc13c1f064f1990d2a1ebe836ffac` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:30.914651 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0186f93-e56f-46ba-8fca-101726cf238a None admin] RBAC: Authorizing `identity:get_project(project_id=d66fc13c1f064f1990d2a1ebe836ffac)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:30.916203 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0186f93-e56f-46ba-8fca-101726cf238a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:30.917853 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 603/1191] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:44:30 2022] GET /identity/v3/projects/d66fc13c1f064f1990d2a1ebe836ffac => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:30.924062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983cbd8-66a4-4b2c-b1bb-6673e12939d0 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:31.008316 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:31.028067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983cbd8-66a4-4b2c-b1bb-6673e12939d0 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:31.065261 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89dad68-8648-4a49-b16b-5aa1a0a5c8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:31.065932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dad68-8648-4a49-b16b-5aa1a0a5c8b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:31.066211 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dad68-8648-4a49-b16b-5aa1a0a5c8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:31.066478 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dad68-8648-4a49-b16b-5aa1a0a5c8b2 None admin] PATH_INFO: `/v3/projects/d66fc13c1f064f1990d2a1ebe836ffac/users/16b73fff7a0b4214b1803a7396b89f6c/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:31.068438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89dad68-8648-4a49-b16b-5aa1a0a5c8b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=d66fc13c1f064f1990d2a1ebe836ffac, user_id=16b73fff7a0b4214b1803a7396b89f6c, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:31.069856 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89dad68-8648-4a49-b16b-5aa1a0a5c8b2 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:31.083894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 589/1192] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:44:30 2022] PUT /identity/v3/projects/d66fc13c1f064f1990d2a1ebe836ffac/users/16b73fff7a0b4214b1803a7396b89f6c/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:44:33.006968 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c59ac-d9c7-4e33-a882-7a94f02f09a2 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:33.007548 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c59ac-d9c7-4e33-a882-7a94f02f09a2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:33.007548 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c59ac-d9c7-4e33-a882-7a94f02f09a2 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:33.008848 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 604/1193] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:44:33.014260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071793b-03ec-482e-af95-c99961f1f414 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:33.014698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071793b-03ec-482e-af95-c99961f1f414 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:33.014808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071793b-03ec-482e-af95-c99961f1f414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:33.033448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-a071793b-03ec-482e-af95-c99961f1f414 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:44:33.034640 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a071793b-03ec-482e-af95-c99961f1f414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:44:33.101385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 590/1194] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:44:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:44:33.372919 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c59ac-d9c7-4e33-a882-7a94f02f09a2 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:33.447543 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:33.481276 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c59ac-d9c7-4e33-a882-7a94f02f09a2 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:33.515721 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b508da9-3f30-4277-bf9a-fe2e9d978892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:33.516379 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b508da9-3f30-4277-bf9a-fe2e9d978892 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:33.516660 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b508da9-3f30-4277-bf9a-fe2e9d978892 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:33.516932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b508da9-3f30-4277-bf9a-fe2e9d978892 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:33.528251 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b508da9-3f30-4277-bf9a-fe2e9d978892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:44:33.532917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-1b508da9-3f30-4277-bf9a-fe2e9d978892 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:44:33.532917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:44:33.532917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:44:33.532917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:44:33.532917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:44:33.532917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:44:33.532917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:44:33.532917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:44:33.532917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:44:33.532917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:44:33.532917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:44:33.532917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:44:33.532917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:44:33.532917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:44:33.532917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:44:33.532917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:44:33.534053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:44:33.534053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:44:33.534053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:44:33.534053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:44:33.534053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:44:33.534053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:44:33.534053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:44:33.534053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:44:33.534053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:44:33.534053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:44:33.534053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:44:33.534053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:44:33.534053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:44:33.534053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:44:33.534053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:44:33.534053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:44:33.535163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:44:33.535163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:44:33.535163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:44:33.535163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:44:33.535163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:44:33.535163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:44:33.535163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:44:33.535163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:44:33.535163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:44:33.535163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:44:33.535163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:44:33.535163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:44:33.535163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 605/1195] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:44:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:44:33.540167 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071793b-03ec-482e-af95-c99961f1f414 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:33.607807 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:33.642190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071793b-03ec-482e-af95-c99961f1f414 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:33.679044 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0af79-bb33-4e1c-b911-831c2e227965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:33.679653 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0af79-bb33-4e1c-b911-831c2e227965 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:33.679933 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0af79-bb33-4e1c-b911-831c2e227965 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:33.680235 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0af79-bb33-4e1c-b911-831c2e227965 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:33.681661 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0af79-bb33-4e1c-b911-831c2e227965 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:44:33.682049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0af79-bb33-4e1c-b911-831c2e227965 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:33.683425 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0af79-bb33-4e1c-b911-831c2e227965 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:33.691622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 591/1196] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:44:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:33.698391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b508da9-3f30-4277-bf9a-fe2e9d978892 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:33.796402 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b508da9-3f30-4277-bf9a-fe2e9d978892 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:33.825577 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:33.831808 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26280226-813e-4043-848f-3870dd1dfb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:33.832469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26280226-813e-4043-848f-3870dd1dfb7a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:33.832742 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26280226-813e-4043-848f-3870dd1dfb7a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:33.833009 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26280226-813e-4043-848f-3870dd1dfb7a None admin] PATH_INFO: `/v3/users/16b73fff7a0b4214b1803a7396b89f6c` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:33.834127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26280226-813e-4043-848f-3870dd1dfb7a None admin] RBAC: Authorizing `identity:get_user(user_id=16b73fff7a0b4214b1803a7396b89f6c)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:33.835550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26280226-813e-4043-848f-3870dd1dfb7a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:33.837289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 606/1197] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 24 08:44:33 2022] GET /identity/v3/users/16b73fff7a0b4214b1803a7396b89f6c => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:33.843428 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0af79-bb33-4e1c-b911-831c2e227965 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:33.944742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0af79-bb33-4e1c-b911-831c2e227965 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:33.974846 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:33.980912 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad49cd7-3d0c-4294-8126-474e676fac02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:33.981525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad49cd7-3d0c-4294-8126-474e676fac02 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:33.981801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad49cd7-3d0c-4294-8126-474e676fac02 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:33.982067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad49cd7-3d0c-4294-8126-474e676fac02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:34.058894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad49cd7-3d0c-4294-8126-474e676fac02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:34.060362 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad49cd7-3d0c-4294-8126-474e676fac02 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:34.159587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 592/1198] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:44:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:44:34.166182 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26280226-813e-4043-848f-3870dd1dfb7a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:34.269500 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26280226-813e-4043-848f-3870dd1dfb7a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:34.306861 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e3ad1-99f4-4b5e-b2b4-f9dec65821d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:34.307610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e3ad1-99f4-4b5e-b2b4-f9dec65821d3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:34.307888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e3ad1-99f4-4b5e-b2b4-f9dec65821d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:34.308186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e3ad1-99f4-4b5e-b2b4-f9dec65821d3 None admin] PATH_INFO: `/v3/projects/d66fc13c1f064f1990d2a1ebe836ffac` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:34.309432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e3ad1-99f4-4b5e-b2b4-f9dec65821d3 None admin] RBAC: Authorizing `identity:get_project(project_id=d66fc13c1f064f1990d2a1ebe836ffac)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:34.310894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e3ad1-99f4-4b5e-b2b4-f9dec65821d3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:34.312629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 607/1199] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 24 08:44:34 2022] GET /identity/v3/projects/d66fc13c1f064f1990d2a1ebe836ffac => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:34.318940 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad49cd7-3d0c-4294-8126-474e676fac02 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:44:34.352086 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:34.424445 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad49cd7-3d0c-4294-8126-474e676fac02 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:44:34.457758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3106ed-bbbd-468f-b7b2-e8ea7c42f887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:44:34.458357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3106ed-bbbd-468f-b7b2-e8ea7c42f887 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:44:34.458631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3106ed-bbbd-468f-b7b2-e8ea7c42f887 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:44:34.458898 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3106ed-bbbd-468f-b7b2-e8ea7c42f887 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:44:34.459617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3106ed-bbbd-468f-b7b2-e8ea7c42f887 None admin] RBAC: Adding query filter params (role.id=b13ffa72344c4856901b1ba003e273fd, scope.project.id=d66fc13c1f064f1990d2a1ebe836ffac, user.id=16b73fff7a0b4214b1803a7396b89f6c) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:44:34.459923 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3106ed-bbbd-468f-b7b2-e8ea7c42f887 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:44:34.461378 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3106ed-bbbd-468f-b7b2-e8ea7c42f887 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:44:34.467778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 593/1200] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:44:34 2022] GET /identity/v3/role_assignments?user.id=16b73fff7a0b4214b1803a7396b89f6c&scope.project.id=d66fc13c1f064f1990d2a1ebe836ffac&role.id=b13ffa72344c4856901b1ba003e273fd => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:44:44.301594 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=67 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:44:44.465094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=52 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:31.410313 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a19fa-4a4e-486e-8a13-e1806405e21b None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:31.410313 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a19fa-4a4e-486e-8a13-e1806405e21b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:31.411005 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a19fa-4a4e-486e-8a13-e1806405e21b None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:31.412877 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 608/1201] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:45:31.418685 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26d1b6-bd83-485b-9e7d-389420dcf741 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:31.419255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26d1b6-bd83-485b-9e7d-389420dcf741 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:31.419255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26d1b6-bd83-485b-9e7d-389420dcf741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:31.442622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-4d26d1b6-bd83-485b-9e7d-389420dcf741 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:45:31.444242 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4d26d1b6-bd83-485b-9e7d-389420dcf741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:45:31.550789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 594/1202] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:45:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:45:31.843127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a19fa-4a4e-486e-8a13-e1806405e21b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:32.003780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a19fa-4a4e-486e-8a13-e1806405e21b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:32.054825 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd299e-54bf-4ed9-815d-acc7f021e718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:32.056047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd299e-54bf-4ed9-815d-acc7f021e718 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:32.056669 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd299e-54bf-4ed9-815d-acc7f021e718 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:32.056952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd299e-54bf-4ed9-815d-acc7f021e718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:32.071997 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd299e-54bf-4ed9-815d-acc7f021e718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:32.074444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd299e-54bf-4ed9-815d-acc7f021e718 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:32.098228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-59bd299e-54bf-4ed9-815d-acc7f021e718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:32.098228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:32.098228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:32.098228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:32.098228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:32.098228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:32.098228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:32.098228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:32.098228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:32.098228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:32.098228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:32.098228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:32.098228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:45:32.098228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:45:32.098228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:45:32.098228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:45:32.099412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:32.099412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:32.099412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:32.099412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:32.099412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:32.099412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:32.099412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:32.099412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:32.099412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:32.099412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:32.099412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:32.099412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:32.099412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:32.099412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:32.099412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:32.099412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:32.099412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:45:32.100433 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:45:32.100433 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:32.100433 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:45:32.100433 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 609/1203] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:45:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:32.106676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d26d1b6-bd83-485b-9e7d-389420dcf741 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:32.241974 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d26d1b6-bd83-485b-9e7d-389420dcf741 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:32.274878 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:32.283482 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c38b4-c6c5-429d-b4f2-50cc050550f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:32.284189 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c38b4-c6c5-429d-b4f2-50cc050550f4 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:32.284473 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c38b4-c6c5-429d-b4f2-50cc050550f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:32.284769 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c38b4-c6c5-429d-b4f2-50cc050550f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:32.285435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c38b4-c6c5-429d-b4f2-50cc050550f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:32.285737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c38b4-c6c5-429d-b4f2-50cc050550f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:32.287512 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c38b4-c6c5-429d-b4f2-50cc050550f4 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:32.302683 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 595/1204] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:32.309280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd299e-54bf-4ed9-815d-acc7f021e718 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:32.386680 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:32.454304 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd299e-54bf-4ed9-815d-acc7f021e718 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:32.511838 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6032a2-fbe7-4b0a-853b-90553252a363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:32.512699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6032a2-fbe7-4b0a-853b-90553252a363 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:32.513102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6032a2-fbe7-4b0a-853b-90553252a363 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:32.513415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6032a2-fbe7-4b0a-853b-90553252a363 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:32.514153 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6032a2-fbe7-4b0a-853b-90553252a363 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:32.515725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6032a2-fbe7-4b0a-853b-90553252a363 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:32.551154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 610/1205] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 24 08:45:32 2022] POST /identity/v3/users => generated 251 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:45:34.850213 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6205f-97f2-44b5-902a-77c100397443 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:34.850798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6205f-97f2-44b5-902a-77c100397443 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:34.851055 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6205f-97f2-44b5-902a-77c100397443 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:34.852638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 596/1206] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:45:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:45:34.859611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b26e0-9159-4f7a-bb76-8f498f68be6f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:34.860332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b26e0-9159-4f7a-bb76-8f498f68be6f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:34.860332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b26e0-9159-4f7a-bb76-8f498f68be6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:34.883444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a54b26e0-9159-4f7a-bb76-8f498f68be6f None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:45:34.884931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a54b26e0-9159-4f7a-bb76-8f498f68be6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:45:34.969968 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 611/1207] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:45:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:45:35.274690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6205f-97f2-44b5-902a-77c100397443 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:35.405910 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6205f-97f2-44b5-902a-77c100397443 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:35.452039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367faec2-7949-427e-9a59-5c5273ade7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:35.453155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367faec2-7949-427e-9a59-5c5273ade7bc None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:35.453889 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367faec2-7949-427e-9a59-5c5273ade7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:35.454405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367faec2-7949-427e-9a59-5c5273ade7bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:35.468221 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367faec2-7949-427e-9a59-5c5273ade7bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:45:35.473025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-367faec2-7949-427e-9a59-5c5273ade7bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:45:35.473025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:35.473025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:35.473025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:35.473025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:35.473025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:35.473025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:35.473025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:35.473025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:35.473025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:35.473025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:35.473025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:35.473025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:45:35.473025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:45:35.473025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:45:35.473025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:45:35.474155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:45:35.474155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:45:35.474155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:45:35.474155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:45:35.474155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:35.474155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:35.474155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:35.474155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:35.474155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:35.474155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:35.474155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:35.474155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:35.474155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:35.474155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:35.474155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:35.474155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:35.475013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:35.475013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:35.475013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:35.475013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:35.475013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:45:35.475013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:45:35.475013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:45:35.475013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:45:35.475013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:45:35.475013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:45:35.475013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:45:35.475013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:45:35.475013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 597/1208] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:45:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:35.480607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b26e0-9159-4f7a-bb76-8f498f68be6f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:35.531581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:35.612211 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b26e0-9159-4f7a-bb76-8f498f68be6f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:35.655925 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb68ced-81e2-4e47-a6ef-a0cfc633e718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:35.656519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb68ced-81e2-4e47-a6ef-a0cfc633e718 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:35.656793 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb68ced-81e2-4e47-a6ef-a0cfc633e718 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:35.657058 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb68ced-81e2-4e47-a6ef-a0cfc633e718 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:35.657729 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb68ced-81e2-4e47-a6ef-a0cfc633e718 None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:35.658029 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb68ced-81e2-4e47-a6ef-a0cfc633e718 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:35.659470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb68ced-81e2-4e47-a6ef-a0cfc633e718 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:35.672136 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 612/1209] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:45:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:35.675656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367faec2-7949-427e-9a59-5c5273ade7bc None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:35.743197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:35.805120 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367faec2-7949-427e-9a59-5c5273ade7bc None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:35.852694 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03364c5-aa62-45e9-934b-6223d0aaa14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:35.855227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03364c5-aa62-45e9-934b-6223d0aaa14f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:35.855227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03364c5-aa62-45e9-934b-6223d0aaa14f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:35.855227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03364c5-aa62-45e9-934b-6223d0aaa14f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:35.866908 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03364c5-aa62-45e9-934b-6223d0aaa14f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:35.868447 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03364c5-aa62-45e9-934b-6223d0aaa14f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:35.887364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-f03364c5-aa62-45e9-934b-6223d0aaa14f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:35.887364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:35.887364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:35.887364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:35.887364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:35.887364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:35.887364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:35.887364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:35.887364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:35.887364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:35.887364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:35.887364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:35.887364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:45:35.887364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:45:35.887364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:45:35.887364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:45:35.888589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:35.888589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:35.888589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:35.888589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:35.888589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:35.888589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:35.888589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:35.888589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:35.888589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:35.888589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:35.888589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:35.888589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:35.888589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:35.888589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:35.888589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:35.888589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:35.888589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:45:35.889476 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:45:35.889476 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:35.889476 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:45:35.889476 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 598/1210] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:45:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:45:35.895463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb68ced-81e2-4e47-a6ef-a0cfc633e718 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:36.024406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:36.028029 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb68ced-81e2-4e47-a6ef-a0cfc633e718 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:36.067650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce66c6-37be-4d3d-8f35-b796f1005d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:36.067650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce66c6-37be-4d3d-8f35-b796f1005d5f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:36.067650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce66c6-37be-4d3d-8f35-b796f1005d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:36.068053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce66c6-37be-4d3d-8f35-b796f1005d5f None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:36.068439 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce66c6-37be-4d3d-8f35-b796f1005d5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:36.068740 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce66c6-37be-4d3d-8f35-b796f1005d5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:36.070105 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce66c6-37be-4d3d-8f35-b796f1005d5f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:36.082591 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 613/1211] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:36.088561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03364c5-aa62-45e9-934b-6223d0aaa14f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:36.205875 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03364c5-aa62-45e9-934b-6223d0aaa14f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:36.207450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:36.244971 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f1180-3b4e-47bc-ac3a-811d91c96227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:36.245909 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f1180-3b4e-47bc-ac3a-811d91c96227 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:36.245909 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f1180-3b4e-47bc-ac3a-811d91c96227 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:36.246239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f1180-3b4e-47bc-ac3a-811d91c96227 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:36.254805 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f1180-3b4e-47bc-ac3a-811d91c96227 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:36.256547 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f1180-3b4e-47bc-ac3a-811d91c96227 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:36.269103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-c47f1180-3b4e-47bc-ac3a-811d91c96227 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:45:36.269103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:36.269103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:36.269103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:36.269103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:36.269103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:36.269103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:36.269103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:36.269103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:36.269103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:36.269103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:36.269103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:36.269103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:45:36.269103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:45:36.269103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:45:36.269103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:45:36.270204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:36.270204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:36.270204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:45:36.270204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:45:36.270204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:45:36.270204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:45:36.270204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:36.270204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:36.270204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:36.270204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:36.270204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:36.270204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:36.270204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:36.270204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:36.270204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:36.270204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:36.270204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:36.270204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:36.271603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:36.271603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:36.271603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:45:36.271603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:45:36.271603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:45:36.271603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:45:36.271603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:45:36.271603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:45:36.271603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:45:36.271603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:45:36.271603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 599/1212] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:45:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:36.276201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce66c6-37be-4d3d-8f35-b796f1005d5f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:36.401232 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce66c6-37be-4d3d-8f35-b796f1005d5f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:36.443098 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2b3a0-3f97-4b6e-921a-9412e08f511c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:36.443749 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2b3a0-3f97-4b6e-921a-9412e08f511c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:36.444003 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2b3a0-3f97-4b6e-921a-9412e08f511c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:36.444320 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2b3a0-3f97-4b6e-921a-9412e08f511c None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:36.445076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2b3a0-3f97-4b6e-921a-9412e08f511c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:36.445383 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2b3a0-3f97-4b6e-921a-9412e08f511c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:36.446813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2b3a0-3f97-4b6e-921a-9412e08f511c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:36.466311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 614/1213] 10.222.0.67 () {56 vars in 1143 bytes} [Wed Aug 24 08:45:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:36.471943 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f1180-3b4e-47bc-ac3a-811d91c96227 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:36.591256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f1180-3b4e-47bc-ac3a-811d91c96227 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:36.633822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945da6e2-cfb9-4bd4-b051-5401a26a855e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:36.634456 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945da6e2-cfb9-4bd4-b051-5401a26a855e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:36.635648 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945da6e2-cfb9-4bd4-b051-5401a26a855e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:36.635648 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945da6e2-cfb9-4bd4-b051-5401a26a855e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:36.670996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:36.712690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945da6e2-cfb9-4bd4-b051-5401a26a855e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:36.715426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945da6e2-cfb9-4bd4-b051-5401a26a855e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:36.858459 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 600/1214] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:45:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:45:36.864360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2b3a0-3f97-4b6e-921a-9412e08f511c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:36.887791 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:36.991068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2b3a0-3f97-4b6e-921a-9412e08f511c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:37.036632 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dfcb8-cae2-4d47-aebd-0bfbef5173eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:37.037273 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dfcb8-cae2-4d47-aebd-0bfbef5173eb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:37.037549 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dfcb8-cae2-4d47-aebd-0bfbef5173eb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:37.037814 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dfcb8-cae2-4d47-aebd-0bfbef5173eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:37.050238 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993dfcb8-cae2-4d47-aebd-0bfbef5173eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:37.051987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993dfcb8-cae2-4d47-aebd-0bfbef5173eb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:37.071077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-993dfcb8-cae2-4d47-aebd-0bfbef5173eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:37.071077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:37.071077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:37.071077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:37.071077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:37.071077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:37.071077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:37.071077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:37.071077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:37.071077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:37.071077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:37.071077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:37.071077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:45:37.071077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:45:37.071077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:45:37.071077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:45:37.072430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:37.072430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:37.072430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:37.072430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:37.072430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:37.072430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:37.072430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:37.072430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:37.072430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:37.072430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:37.072430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:37.072430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:37.072430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:37.072430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:37.072430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:37.072430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:37.072430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:45:37.073481 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:45:37.073481 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:37.073481 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:45:37.073481 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 615/1215] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:45:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:45:37.078537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945da6e2-cfb9-4bd4-b051-5401a26a855e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:37.147703 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:37.192465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945da6e2-cfb9-4bd4-b051-5401a26a855e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:37.232572 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7dcda-1c37-4e4a-87c2-355d280e15ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:37.233326 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7dcda-1c37-4e4a-87c2-355d280e15ef None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:37.233690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7dcda-1c37-4e4a-87c2-355d280e15ef None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:37.233998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7dcda-1c37-4e4a-87c2-355d280e15ef None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:37.234953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7dcda-1c37-4e4a-87c2-355d280e15ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:37.235257 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7dcda-1c37-4e4a-87c2-355d280e15ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:37.236756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7dcda-1c37-4e4a-87c2-355d280e15ef None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:37.252223 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 601/1216] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:37.256875 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dfcb8-cae2-4d47-aebd-0bfbef5173eb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:37.348779 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:37.397806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dfcb8-cae2-4d47-aebd-0bfbef5173eb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:37.442929 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a55e9f-79fb-477e-b3e5-bcc6d762a805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:37.443573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a55e9f-79fb-477e-b3e5-bcc6d762a805 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:37.443847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a55e9f-79fb-477e-b3e5-bcc6d762a805 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:37.444141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a55e9f-79fb-477e-b3e5-bcc6d762a805 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:37.452249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a55e9f-79fb-477e-b3e5-bcc6d762a805 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:37.453906 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a55e9f-79fb-477e-b3e5-bcc6d762a805 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:37.468047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-66a55e9f-79fb-477e-b3e5-bcc6d762a805 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:45:37.468047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:37.468047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:37.468047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:37.468047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:37.468047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:37.468047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:37.468047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:37.468047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:37.468047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:37.468047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:37.468047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:37.468047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:45:37.468047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:45:37.468047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:45:37.469096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:45:37.469096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:37.469096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:37.469096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:37.469096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:37.469096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:37.469096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:37.469096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:37.469096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:37.469096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:37.469096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:37.469096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:37.469096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:37.469096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:37.469096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:37.469096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:37.469096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:37.470021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:45:37.470021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:45:37.470021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:45:37.470021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:45:37.470021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:45:37.470021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:45:37.470021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:45:37.470021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:45:37.472462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 616/1217] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:45:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:37.475173 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7dcda-1c37-4e4a-87c2-355d280e15ef None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:37.600559 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7dcda-1c37-4e4a-87c2-355d280e15ef None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:37.624465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:37.649667 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f8e48-e954-48a1-b68b-af6cacb47a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:37.650627 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f8e48-e954-48a1-b68b-af6cacb47a48 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:37.651051 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f8e48-e954-48a1-b68b-af6cacb47a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:37.651176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f8e48-e954-48a1-b68b-af6cacb47a48 None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:37.652670 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f8e48-e954-48a1-b68b-af6cacb47a48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:37.652979 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f8e48-e954-48a1-b68b-af6cacb47a48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:37.654462 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f8e48-e954-48a1-b68b-af6cacb47a48 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:37.669680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 602/1218] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:45:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:37.675951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a55e9f-79fb-477e-b3e5-bcc6d762a805 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:37.807628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a55e9f-79fb-477e-b3e5-bcc6d762a805 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:37.847104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:37.857827 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b218a5-61d5-4796-9a46-5f8559e79151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:37.858741 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b218a5-61d5-4796-9a46-5f8559e79151 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:37.859196 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b218a5-61d5-4796-9a46-5f8559e79151 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:37.859585 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b218a5-61d5-4796-9a46-5f8559e79151 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:37.860809 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b218a5-61d5-4796-9a46-5f8559e79151 None admin] RBAC: Adding query filter params (role.id=9aa748c8ea2542e6987e0b93190ea4e6, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=33a0844a1c8f4ea4932e12d53e5ee9ef) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:37.861427 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b218a5-61d5-4796-9a46-5f8559e79151 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:37.864360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b218a5-61d5-4796-9a46-5f8559e79151 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:37.875457 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 617/1219] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:45:37 2022] GET /identity/v3/role_assignments?user.id=33a0844a1c8f4ea4932e12d53e5ee9ef&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=9aa748c8ea2542e6987e0b93190ea4e6 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:39.916116 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab23b6-0658-425e-925d-5bcf24f7bc0a None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:39.916785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab23b6-0658-425e-925d-5bcf24f7bc0a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:39.916785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab23b6-0658-425e-925d-5bcf24f7bc0a None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:39.918888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 603/1220] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:45:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:45:39.925163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45527f82-9c71-40fa-967c-8be59dc0d7d2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:39.925751 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45527f82-9c71-40fa-967c-8be59dc0d7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:39.925751 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45527f82-9c71-40fa-967c-8be59dc0d7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:39.946988 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-45527f82-9c71-40fa-967c-8be59dc0d7d2 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:45:39.948311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-45527f82-9c71-40fa-967c-8be59dc0d7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:45:40.032519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 618/1221] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:45:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:45:40.358374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab23b6-0658-425e-925d-5bcf24f7bc0a None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:40.514984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab23b6-0658-425e-925d-5bcf24f7bc0a None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:40.562256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d47a2-cfd9-4b44-bafa-43c536a991c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:40.562836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d47a2-cfd9-4b44-bafa-43c536a991c5 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:40.562836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d47a2-cfd9-4b44-bafa-43c536a991c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:40.563032 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d47a2-cfd9-4b44-bafa-43c536a991c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:40.575596 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d47a2-cfd9-4b44-bafa-43c536a991c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:45:40.580568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-d62d47a2-cfd9-4b44-bafa-43c536a991c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:45:40.580568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:40.580568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:40.580568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:40.580568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:40.580568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:40.580568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:40.580568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:40.580568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:40.580568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:40.580568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:40.580568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:40.580568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:45:40.580568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:45:40.580568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:45:40.580568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:45:40.581876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:45:40.581876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:45:40.581876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:45:40.581876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:45:40.581876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:40.581876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:40.581876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:40.581876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:40.581876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:40.581876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:40.581876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:40.581876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:40.581876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:40.581876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:40.581876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:40.581876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:40.582932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:40.582932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:40.582932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:40.582932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:40.582932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:45:40.582932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:45:40.582932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:45:40.582932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:45:40.582932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:45:40.582932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:45:40.582932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:45:40.582932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:45:40.582932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 604/1222] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:45:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:40.591971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45527f82-9c71-40fa-967c-8be59dc0d7d2 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:40.723026 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45527f82-9c71-40fa-967c-8be59dc0d7d2 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:40.754797 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:40.770353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddadca-ffc0-4764-9a1b-07d240ffe247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:40.771336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddadca-ffc0-4764-9a1b-07d240ffe247 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:40.771838 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddadca-ffc0-4764-9a1b-07d240ffe247 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:40.772207 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddadca-ffc0-4764-9a1b-07d240ffe247 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:40.773129 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddadca-ffc0-4764-9a1b-07d240ffe247 None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:40.773488 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddadca-ffc0-4764-9a1b-07d240ffe247 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:40.775372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddadca-ffc0-4764-9a1b-07d240ffe247 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:40.788614 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 619/1223] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:45:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:40.796316 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d47a2-cfd9-4b44-bafa-43c536a991c5 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:40.858732 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:40.931562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d47a2-cfd9-4b44-bafa-43c536a991c5 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:40.969953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99e5e5-3d5d-458f-b3ea-a314b342441f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:40.970596 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99e5e5-3d5d-458f-b3ea-a314b342441f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:40.970914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99e5e5-3d5d-458f-b3ea-a314b342441f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:40.971145 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99e5e5-3d5d-458f-b3ea-a314b342441f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:40.984207 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b99e5e5-3d5d-458f-b3ea-a314b342441f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:40.986131 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b99e5e5-3d5d-458f-b3ea-a314b342441f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:41.004509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-8b99e5e5-3d5d-458f-b3ea-a314b342441f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:41.004509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:41.004509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:41.004509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:41.004509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:41.004509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:41.004509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:41.004509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:41.004509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:41.004509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:41.004509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:41.004509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:41.004509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:45:41.004509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:45:41.004509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:45:41.004509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:45:41.006455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:41.006455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:41.006455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:41.006455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:41.006455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:41.006455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:41.006455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:41.006455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:41.006455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:41.006455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:41.006455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:41.006455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:41.006455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:41.006455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:41.006455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:41.006455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:41.006455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:45:41.008312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:45:41.008312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:41.008312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:45:41.008312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 605/1224] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:45:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:41.012424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddadca-ffc0-4764-9a1b-07d240ffe247 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:41.126599 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddadca-ffc0-4764-9a1b-07d240ffe247 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:41.164715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75a576-02df-463e-b01c-635c4966bc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:41.165386 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75a576-02df-463e-b01c-635c4966bc9d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:41.165671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75a576-02df-463e-b01c-635c4966bc9d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:41.165938 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75a576-02df-463e-b01c-635c4966bc9d None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:41.166580 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75a576-02df-463e-b01c-635c4966bc9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:41.166880 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75a576-02df-463e-b01c-635c4966bc9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:41.169066 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75a576-02df-463e-b01c-635c4966bc9d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:41.183099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 620/1225] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:41.190083 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99e5e5-3d5d-458f-b3ea-a314b342441f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:41.303781 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:41.311795 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99e5e5-3d5d-458f-b3ea-a314b342441f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:41.358289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c93f2-4267-4c74-9d9c-c7f4eaa1928f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:41.358925 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c93f2-4267-4c74-9d9c-c7f4eaa1928f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:41.359208 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c93f2-4267-4c74-9d9c-c7f4eaa1928f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:41.359476 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c93f2-4267-4c74-9d9c-c7f4eaa1928f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:41.368730 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c93f2-4267-4c74-9d9c-c7f4eaa1928f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:41.370179 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c93f2-4267-4c74-9d9c-c7f4eaa1928f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:41.383226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-2d7c93f2-4267-4c74-9d9c-c7f4eaa1928f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:45:41.383226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:41.383226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:41.383226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:41.383226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:41.383226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:41.383226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:41.383226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:41.383226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:41.383226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:41.383226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:41.383226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:41.383226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:45:41.383226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:45:41.383226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:45:41.383226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:45:41.385351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:41.385351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:41.385351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:45:41.385351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:45:41.385351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:45:41.385351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:45:41.385351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:41.385351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:41.385351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:41.385351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:41.385351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:41.385351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:41.385351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:41.385351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:41.385351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:41.385351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:41.385351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:41.385351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:41.387917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:41.387917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:41.387917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:45:41.387917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:45:41.387917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:45:41.387917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:45:41.387917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:45:41.387917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:45:41.387917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:45:41.387917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:45:41.387917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 606/1226] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:45:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:41.391446 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75a576-02df-463e-b01c-635c4966bc9d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:41.446644 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:41.507098 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75a576-02df-463e-b01c-635c4966bc9d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:41.544867 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ce61f-dccd-4aa7-a375-8da76751966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:41.545508 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ce61f-dccd-4aa7-a375-8da76751966b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:41.545794 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ce61f-dccd-4aa7-a375-8da76751966b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:41.546071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ce61f-dccd-4aa7-a375-8da76751966b None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:41.546827 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ce61f-dccd-4aa7-a375-8da76751966b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:41.547112 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ce61f-dccd-4aa7-a375-8da76751966b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:41.548664 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ce61f-dccd-4aa7-a375-8da76751966b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:41.569415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 621/1227] 10.222.0.67 () {56 vars in 1143 bytes} [Wed Aug 24 08:45:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:41.577263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c93f2-4267-4c74-9d9c-c7f4eaa1928f None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:41.703325 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c93f2-4267-4c74-9d9c-c7f4eaa1928f None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:41.749109 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9dd7bc-e3b3-4f56-ac3a-389477f77b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:41.749754 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9dd7bc-e3b3-4f56-ac3a-389477f77b84 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:41.750034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9dd7bc-e3b3-4f56-ac3a-389477f77b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:41.750305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9dd7bc-e3b3-4f56-ac3a-389477f77b84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:41.762052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9dd7bc-e3b3-4f56-ac3a-389477f77b84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:41.763530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9dd7bc-e3b3-4f56-ac3a-389477f77b84 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:41.779708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-5b9dd7bc-e3b3-4f56-ac3a-389477f77b84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:41.779708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:41.779708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:41.779708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:41.779708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:41.779708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:41.779708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:41.779708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:41.779708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:41.779708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:41.779708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:41.779708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:41.779708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:45:41.779708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:45:41.779708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:45:41.779708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:45:41.780985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:41.780985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:41.780985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:41.780985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:41.780985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:41.780985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:41.780985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:41.780985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:41.780985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:41.780985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:41.780985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:41.780985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:41.780985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:41.780985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:41.780985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:41.780985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:41.780985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:45:41.782089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:45:41.782089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:41.782089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:45:41.784305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 607/1228] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:45:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:41.787300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ce61f-dccd-4aa7-a375-8da76751966b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:41.913390 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ce61f-dccd-4aa7-a375-8da76751966b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:41.915862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:41.953834 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25667f78-52ec-4461-b52e-0f384308d595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:41.954445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25667f78-52ec-4461-b52e-0f384308d595 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:41.954720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25667f78-52ec-4461-b52e-0f384308d595 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:41.954981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25667f78-52ec-4461-b52e-0f384308d595 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:41.955677 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25667f78-52ec-4461-b52e-0f384308d595 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:41.956178 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25667f78-52ec-4461-b52e-0f384308d595 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:41.957599 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25667f78-52ec-4461-b52e-0f384308d595 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:41.975970 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 622/1229] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:41.986643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9dd7bc-e3b3-4f56-ac3a-389477f77b84 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:41.998244 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:42.098429 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9dd7bc-e3b3-4f56-ac3a-389477f77b84 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:42.141038 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c47e05-786e-4d50-bd9e-6fd8529595e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:42.141828 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c47e05-786e-4d50-bd9e-6fd8529595e9 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:42.142097 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c47e05-786e-4d50-bd9e-6fd8529595e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:42.142350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c47e05-786e-4d50-bd9e-6fd8529595e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:42.150340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c47e05-786e-4d50-bd9e-6fd8529595e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:42.153079 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c47e05-786e-4d50-bd9e-6fd8529595e9 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:42.166100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-a3c47e05-786e-4d50-bd9e-6fd8529595e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:45:42.166100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:42.166100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:42.166100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:42.166100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:42.166100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:42.166100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:42.166100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:42.166100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:42.166100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:42.166100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:42.166100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:42.166100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:45:42.166100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:45:42.166100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:45:42.172124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:45:42.172124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:42.172124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:42.172124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:42.172124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:42.172124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:42.172124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:42.172124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:42.172124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:42.172124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:42.172124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:42.172124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:42.172124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:42.172124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:42.172124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:42.172124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:42.172124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:42.173273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:45:42.173273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:45:42.173273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:45:42.173273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:45:42.173273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:45:42.173273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:45:42.173273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:45:42.173273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:45:42.173273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 608/1230] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:45:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:42.173853 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25667f78-52ec-4461-b52e-0f384308d595 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:42.285477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25667f78-52ec-4461-b52e-0f384308d595 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:42.328245 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50751f9a-4f8d-4553-85ec-2372b9196639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:42.328931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50751f9a-4f8d-4553-85ec-2372b9196639 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:42.329274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50751f9a-4f8d-4553-85ec-2372b9196639 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:42.329573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50751f9a-4f8d-4553-85ec-2372b9196639 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:42.330343 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50751f9a-4f8d-4553-85ec-2372b9196639 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:42.330869 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50751f9a-4f8d-4553-85ec-2372b9196639 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:42.332680 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50751f9a-4f8d-4553-85ec-2372b9196639 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:42.348677 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 623/1231] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:45:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:45:42.355332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c47e05-786e-4d50-bd9e-6fd8529595e9 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:42.434423 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:42.472302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c47e05-786e-4d50-bd9e-6fd8529595e9 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:42.509338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65177d95-1ca9-4c09-a32f-7fc1dcd45134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:42.510013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65177d95-1ca9-4c09-a32f-7fc1dcd45134 None admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:42.510323 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65177d95-1ca9-4c09-a32f-7fc1dcd45134 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:42.510577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65177d95-1ca9-4c09-a32f-7fc1dcd45134 None admin] PATH_INFO: `/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/33a0844a1c8f4ea4932e12d53e5ee9ef/roles/9aa748c8ea2542e6987e0b93190ea4e6` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:42.527625 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65177d95-1ca9-4c09-a32f-7fc1dcd45134 None admin] RBAC: Authorizing `identity:create_grant(project_id=fce6ee19f58b4a468c9a5d169e29a018, user_id=33a0844a1c8f4ea4932e12d53e5ee9ef, role_id=9aa748c8ea2542e6987e0b93190ea4e6)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:42.529682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65177d95-1ca9-4c09-a32f-7fc1dcd45134 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:42.545908 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 609/1232] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:45:42 2022] PUT /identity/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/33a0844a1c8f4ea4932e12d53e5ee9ef/roles/9aa748c8ea2542e6987e0b93190ea4e6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:45:44.506569 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543e6dd-518a-43fd-bf28-2ed2f7ee09c2 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:44.507167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543e6dd-518a-43fd-bf28-2ed2f7ee09c2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:44.507261 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543e6dd-518a-43fd-bf28-2ed2f7ee09c2 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:44.508845 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 624/1233] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:45:44.514580 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d26a3-f68b-4205-8d8c-d64d7c82e7ef None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:44.515060 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d26a3-f68b-4205-8d8c-d64d7c82e7ef None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:44.515239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d26a3-f68b-4205-8d8c-d64d7c82e7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:44.533732 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-dc2d26a3-f68b-4205-8d8c-d64d7c82e7ef None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:45:44.534953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dc2d26a3-f68b-4205-8d8c-d64d7c82e7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:45:44.604744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 610/1234] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:45:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:45:44.882152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543e6dd-518a-43fd-bf28-2ed2f7ee09c2 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:44.908413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:45.007893 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543e6dd-518a-43fd-bf28-2ed2f7ee09c2 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:45.049334 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78993f7d-2be7-4803-b6a5-fafdc2e4dfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:45.049977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78993f7d-2be7-4803-b6a5-fafdc2e4dfa3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:45.050259 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78993f7d-2be7-4803-b6a5-fafdc2e4dfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:45.050533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78993f7d-2be7-4803-b6a5-fafdc2e4dfa3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:45.062235 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78993f7d-2be7-4803-b6a5-fafdc2e4dfa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:45:45.066887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-78993f7d-2be7-4803-b6a5-fafdc2e4dfa3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:45:45.066887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:45.066887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:45.066887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:45.066887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:45.066887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:45.066887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:45.066887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:45.066887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:45.066887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:45.066887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:45.066887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:45.066887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:45:45.066887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:45:45.066887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:45:45.066887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:45:45.068042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:45:45.068042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:45:45.068042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:45:45.068042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:45:45.068042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:45.068042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:45.068042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:45.068042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:45.068042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:45.068042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:45.068042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:45.068042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:45.068042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:45.068042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:45.068042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:45.068042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:45.070191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:45.070191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:45.070191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:45.070191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:45.070191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:45:45.070191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:45:45.070191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:45:45.070191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:45:45.070191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:45:45.070191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:45:45.070191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 08:45:45.070191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:45:45.070191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 625/1235] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 24 08:45:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 08:45:45.074708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d26a3-f68b-4205-8d8c-d64d7c82e7ef None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:45.133267 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:45.204353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d26a3-f68b-4205-8d8c-d64d7c82e7ef None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:45.257745 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c25e6-c0ee-4a77-b962-0fa28c55346c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:45.258583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c25e6-c0ee-4a77-b962-0fa28c55346c None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:45.258902 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c25e6-c0ee-4a77-b962-0fa28c55346c None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:45.259178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c25e6-c0ee-4a77-b962-0fa28c55346c None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:45.259939 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c25e6-c0ee-4a77-b962-0fa28c55346c None admin] RBAC: Adding query filter params (name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:45.260278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c25e6-c0ee-4a77-b962-0fa28c55346c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:45.262892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c25e6-c0ee-4a77-b962-0fa28c55346c None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:45.272993 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 611/1236] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 24 08:45:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:45.279525 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78993f7d-2be7-4803-b6a5-fafdc2e4dfa3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:45.376413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:45.395836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78993f7d-2be7-4803-b6a5-fafdc2e4dfa3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:45.436901 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb68be-35d8-4ed6-8393-e6043b69948d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:45.437720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb68be-35d8-4ed6-8393-e6043b69948d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:45.437962 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb68be-35d8-4ed6-8393-e6043b69948d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:45.438208 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb68be-35d8-4ed6-8393-e6043b69948d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:45.449572 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb68be-35d8-4ed6-8393-e6043b69948d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:45.451227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb68be-35d8-4ed6-8393-e6043b69948d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:45.469299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-48cb68be-35d8-4ed6-8393-e6043b69948d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:45.469299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:45.469299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:45.469299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:45.469299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:45.469299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:45.469299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:45.469299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:45.469299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:45.469299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:45.469299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:45.469299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:45.469299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:45:45.469299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:45:45.469299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:45:45.469299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:45:45.470370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:45.470370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:45.470370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:45.470370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:45.470370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:45.470370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:45.470370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:45.470370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:45.470370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:45.470370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:45.470370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:45.470370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:45.470370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:45.470370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:45.470370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:45.470370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:45.470370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:45:45.471290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:45:45.471290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:45.471290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:45:45.471290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 626/1237] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:45:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:45.476748 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c25e6-c0ee-4a77-b962-0fa28c55346c None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:45.595053 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c25e6-c0ee-4a77-b962-0fa28c55346c None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:45.604595 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:45.637886 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaaa5b2-56bc-49dd-8c48-b7c2d47f4b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:45.638697 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaa5b2-56bc-49dd-8c48-b7c2d47f4b4e None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:45.639217 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaa5b2-56bc-49dd-8c48-b7c2d47f4b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:45.639603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaa5b2-56bc-49dd-8c48-b7c2d47f4b4e None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:45.640430 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaaa5b2-56bc-49dd-8c48-b7c2d47f4b4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:45.640767 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaaa5b2-56bc-49dd-8c48-b7c2d47f4b4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:45.642862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaaa5b2-56bc-49dd-8c48-b7c2d47f4b4e None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:45.663818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 612/1238] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:45.665485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb68be-35d8-4ed6-8393-e6043b69948d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:45.785423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb68be-35d8-4ed6-8393-e6043b69948d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:45.826998 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4cc8c-bab5-4ed1-97b2-b1f98d80c203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:45.827836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4cc8c-bab5-4ed1-97b2-b1f98d80c203 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:45.828683 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4cc8c-bab5-4ed1-97b2-b1f98d80c203 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:45.828683 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4cc8c-bab5-4ed1-97b2-b1f98d80c203 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:45.832463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:45.841366 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d4cc8c-bab5-4ed1-97b2-b1f98d80c203 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:45.843050 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d4cc8c-bab5-4ed1-97b2-b1f98d80c203 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:45.856974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-53d4cc8c-bab5-4ed1-97b2-b1f98d80c203 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:45:45.856974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:45.856974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:45.856974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:45.856974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:45.856974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:45.856974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:45.856974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:45.856974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:45.856974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:45.856974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:45.856974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:45.856974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:45:45.856974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:45:45.856974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:45:45.856974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:45:45.858249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:45.858249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:45.858249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:45:45.858249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:45:45.858249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:45:45.858249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:45:45.858249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:45.858249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:45.858249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:45.858249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:45.858249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:45.858249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:45.858249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:45.858249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:45.858249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:45.858249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:45.858249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:45.858249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:45.859242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:45.859242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:45.859242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:45:45.859242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:45:45.859242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:45:45.859242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:45:45.859242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:45:45.859242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:45:45.859242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:45:45.859242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:45:45.859242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 627/1239] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:45:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:45.864188 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaaa5b2-56bc-49dd-8c48-b7c2d47f4b4e None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:45.979623 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaaa5b2-56bc-49dd-8c48-b7c2d47f4b4e None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:46.019793 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb403e-d84e-4442-baee-654111406780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:46.020705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb403e-d84e-4442-baee-654111406780 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:46.020986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb403e-d84e-4442-baee-654111406780 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:46.021263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb403e-d84e-4442-baee-654111406780 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:46.022003 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb403e-d84e-4442-baee-654111406780 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:46.022459 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb403e-d84e-4442-baee-654111406780 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:46.023899 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb403e-d84e-4442-baee-654111406780 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:46.043507 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 613/1240] 10.222.0.67 () {56 vars in 1143 bytes} [Wed Aug 24 08:45:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:46.049428 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4cc8c-bab5-4ed1-97b2-b1f98d80c203 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:46.195868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4cc8c-bab5-4ed1-97b2-b1f98d80c203 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:46.239418 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cde643-e6e8-4ded-8023-367a7402b63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:46.239944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cde643-e6e8-4ded-8023-367a7402b63a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:46.240254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cde643-e6e8-4ded-8023-367a7402b63a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:46.240525 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cde643-e6e8-4ded-8023-367a7402b63a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:46.312162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:46.321107 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cde643-e6e8-4ded-8023-367a7402b63a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:46.323080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cde643-e6e8-4ded-8023-367a7402b63a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:46.438506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 628/1241] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:45:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 08:45:46.444568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb403e-d84e-4442-baee-654111406780 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:46.475461 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:46.559844 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb403e-d84e-4442-baee-654111406780 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:46.598343 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab000889-9f97-421a-be22-380c59358336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:46.599020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab000889-9f97-421a-be22-380c59358336 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:46.599293 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab000889-9f97-421a-be22-380c59358336 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:46.599642 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab000889-9f97-421a-be22-380c59358336 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:46.616072 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab000889-9f97-421a-be22-380c59358336 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:46.618178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab000889-9f97-421a-be22-380c59358336 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:46.642318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-ab000889-9f97-421a-be22-380c59358336 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:46.642318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:46.642318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:46.642318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:46.642318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:46.642318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:46.642318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:46.642318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:46.642318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:46.642318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:46.642318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:46.642318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:46.642318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:45:46.642318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:45:46.642318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:45:46.642318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:45:46.643802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:46.643802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:46.643802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:46.643802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:46.643802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:46.643802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:46.643802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:46.643802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:46.643802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:46.643802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:46.643802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:46.643802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:46.643802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:46.643802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:46.643802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:46.643802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:46.643802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:45:46.644757 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:45:46.644757 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:46.644757 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:45:46.647773 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 614/1242] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:45:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:46.654138 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cde643-e6e8-4ded-8023-367a7402b63a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:46.772986 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:46.780122 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cde643-e6e8-4ded-8023-367a7402b63a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:46.821936 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30527c5-a28c-48c7-90d6-978a24944bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:46.822480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30527c5-a28c-48c7-90d6-978a24944bff None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:46.822703 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30527c5-a28c-48c7-90d6-978a24944bff None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:46.822976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30527c5-a28c-48c7-90d6-978a24944bff None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:46.823629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30527c5-a28c-48c7-90d6-978a24944bff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:46.824705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30527c5-a28c-48c7-90d6-978a24944bff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:46.826086 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30527c5-a28c-48c7-90d6-978a24944bff None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:46.838406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 629/1243] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:46.848072 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab000889-9f97-421a-be22-380c59358336 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:46.943523 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:46.969232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab000889-9f97-421a-be22-380c59358336 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:47.006975 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedaf32-a54d-477b-b013-61135cb53694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:47.007813 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedaf32-a54d-477b-b013-61135cb53694 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:47.008185 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedaf32-a54d-477b-b013-61135cb53694 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:47.008458 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedaf32-a54d-477b-b013-61135cb53694 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:47.017791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedaf32-a54d-477b-b013-61135cb53694 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:47.019414 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedaf32-a54d-477b-b013-61135cb53694 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:47.034791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-3aedaf32-a54d-477b-b013-61135cb53694 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:45:47.034791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:47.034791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:47.034791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:47.034791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:47.034791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:47.034791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:47.034791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:47.034791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:47.034791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:47.034791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:47.034791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:47.034791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:45:47.034791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:45:47.034791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:45:47.035977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:45:47.035977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:47.035977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:47.035977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:47.035977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:47.035977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:47.035977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:47.035977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:47.035977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:47.035977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:47.035977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:47.035977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:47.035977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:47.035977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:47.035977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:47.035977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:47.035977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:47.037395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:45:47.037395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:45:47.037395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:45:47.037395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:45:47.037395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:45:47.037395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:45:47.037395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:45:47.037395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:45:47.037395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 615/1244] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:45:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:47.042422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30527c5-a28c-48c7-90d6-978a24944bff None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:47.164359 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30527c5-a28c-48c7-90d6-978a24944bff None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:47.204299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115be3a1-bbee-4ece-9d77-644cedf1e17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:47.204918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115be3a1-bbee-4ece-9d77-644cedf1e17c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:47.205279 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115be3a1-bbee-4ece-9d77-644cedf1e17c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:47.205573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115be3a1-bbee-4ece-9d77-644cedf1e17c None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:47.206322 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115be3a1-bbee-4ece-9d77-644cedf1e17c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:47.206612 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115be3a1-bbee-4ece-9d77-644cedf1e17c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:47.208234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115be3a1-bbee-4ece-9d77-644cedf1e17c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:47.222777 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 630/1245] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:45:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:47.229338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedaf32-a54d-477b-b013-61135cb53694 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:47.356538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedaf32-a54d-477b-b013-61135cb53694 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:47.399574 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514db42-6e98-4f5d-8ada-1513d585192f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:47.400614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514db42-6e98-4f5d-8ada-1513d585192f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:47.401004 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514db42-6e98-4f5d-8ada-1513d585192f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:47.401283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514db42-6e98-4f5d-8ada-1513d585192f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:47.402344 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514db42-6e98-4f5d-8ada-1513d585192f None admin] RBAC: Adding query filter params (role.id=9aa748c8ea2542e6987e0b93190ea4e6, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=33a0844a1c8f4ea4932e12d53e5ee9ef) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:47.402900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514db42-6e98-4f5d-8ada-1513d585192f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:47.405168 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514db42-6e98-4f5d-8ada-1513d585192f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:47.415847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 616/1246] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:45:47 2022] GET /identity/v3/role_assignments?user.id=33a0844a1c8f4ea4932e12d53e5ee9ef&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=9aa748c8ea2542e6987e0b93190ea4e6 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:49.594049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8fbe1-1ef6-4c3d-982b-c1c1e57e10f1 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:49.594620 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8fbe1-1ef6-4c3d-982b-c1c1e57e10f1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:49.594620 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8fbe1-1ef6-4c3d-982b-c1c1e57e10f1 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:49.596148 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 631/1247] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:45:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:45:49.602415 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c509fd-7c43-4af0-b357-451a86fa20f9 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:49.602913 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c509fd-7c43-4af0-b357-451a86fa20f9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:49.603144 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c509fd-7c43-4af0-b357-451a86fa20f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:49.615245 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:49.632053 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d3c509fd-7c43-4af0-b357-451a86fa20f9 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:45:49.633377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d3c509fd-7c43-4af0-b357-451a86fa20f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:45:49.719888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 617/1248] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:45:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:45:50.001670 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8fbe1-1ef6-4c3d-982b-c1c1e57e10f1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:50.023289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:50.124616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8fbe1-1ef6-4c3d-982b-c1c1e57e10f1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:50.164279 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5258b5-5ee1-4feb-815c-0899c4f7d1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:50.164900 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5258b5-5ee1-4feb-815c-0899c4f7d1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:50.165169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5258b5-5ee1-4feb-815c-0899c4f7d1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:50.165446 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5258b5-5ee1-4feb-815c-0899c4f7d1b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:50.177793 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5258b5-5ee1-4feb-815c-0899c4f7d1b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:45:50.183099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-bf5258b5-5ee1-4feb-815c-0899c4f7d1b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:45:50.183099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:50.183099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:50.183099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:50.183099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:50.183099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:50.183099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:50.183099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:50.183099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:50.183099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:50.183099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:50.183099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:50.183099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:45:50.183099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:45:50.183099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:45:50.183099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:45:50.185198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:45:50.185198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:45:50.185198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:45:50.185198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:45:50.185198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:50.185198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:50.185198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:50.185198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:50.185198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:50.185198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:50.185198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:50.185198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:50.185198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:50.185198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:50.185198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:50.185198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:50.186978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:50.186978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:50.186978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:50.186978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:50.186978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:45:50.186978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:45:50.186978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:45:50.186978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:45:50.186978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:45:50.186978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:45:50.186978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:45:50.186978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:45:50.186978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 632/1249] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:45:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:50.192161 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c509fd-7c43-4af0-b357-451a86fa20f9 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:50.331626 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c509fd-7c43-4af0-b357-451a86fa20f9 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:50.379011 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:50.385261 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928cfbb-3cbf-4a8a-96b6-ca39e35882ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:50.386325 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928cfbb-3cbf-4a8a-96b6-ca39e35882ab None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:50.387355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928cfbb-3cbf-4a8a-96b6-ca39e35882ab None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:50.387784 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928cfbb-3cbf-4a8a-96b6-ca39e35882ab None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:50.388862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928cfbb-3cbf-4a8a-96b6-ca39e35882ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:50.389543 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928cfbb-3cbf-4a8a-96b6-ca39e35882ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:50.390953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928cfbb-3cbf-4a8a-96b6-ca39e35882ab None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:50.400160 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 618/1250] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:45:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:45:50.408020 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5258b5-5ee1-4feb-815c-0899c4f7d1b5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:50.508732 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:50.539532 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5258b5-5ee1-4feb-815c-0899c4f7d1b5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:50.579985 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b33d42-6af3-4199-b112-e5aa592af302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:50.580949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b33d42-6af3-4199-b112-e5aa592af302 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:50.581211 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b33d42-6af3-4199-b112-e5aa592af302 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:50.581484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b33d42-6af3-4199-b112-e5aa592af302 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:50.594384 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b33d42-6af3-4199-b112-e5aa592af302 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:50.595789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b33d42-6af3-4199-b112-e5aa592af302 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:50.612583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-39b33d42-6af3-4199-b112-e5aa592af302 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:50.612583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:50.612583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:50.612583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:50.612583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:50.612583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:50.612583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:50.612583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:50.612583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:50.612583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:50.612583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:50.612583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:50.612583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:45:50.612583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:45:50.612583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:45:50.612583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:45:50.623789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:50.623789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:50.623789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:50.623789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:50.623789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:50.623789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:50.623789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:50.623789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:50.623789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:50.623789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:50.623789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:50.623789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:50.623789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:50.623789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:50.623789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:50.623789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:50.623789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:45:50.625660 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:45:50.625660 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:50.625660 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:45:50.625660 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 633/1251] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:45:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:50.626111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928cfbb-3cbf-4a8a-96b6-ca39e35882ab None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:50.765728 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928cfbb-3cbf-4a8a-96b6-ca39e35882ab None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:50.809788 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467988c8-021b-41fa-85de-b35ea46375fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:50.810431 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467988c8-021b-41fa-85de-b35ea46375fa None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:50.810729 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467988c8-021b-41fa-85de-b35ea46375fa None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:50.811035 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467988c8-021b-41fa-85de-b35ea46375fa None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:50.811956 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467988c8-021b-41fa-85de-b35ea46375fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:50.812317 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467988c8-021b-41fa-85de-b35ea46375fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:50.814300 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467988c8-021b-41fa-85de-b35ea46375fa None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:50.828456 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 619/1252] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:50.834643 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b33d42-6af3-4199-b112-e5aa592af302 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:50.952379 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b33d42-6af3-4199-b112-e5aa592af302 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:50.986401 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:50.994333 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc89b66-82d1-4b79-8e88-0c8d7e10a8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:50.995031 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc89b66-82d1-4b79-8e88-0c8d7e10a8c6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:50.995456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc89b66-82d1-4b79-8e88-0c8d7e10a8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:50.995759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc89b66-82d1-4b79-8e88-0c8d7e10a8c6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:51.005800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc89b66-82d1-4b79-8e88-0c8d7e10a8c6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:51.007288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc89b66-82d1-4b79-8e88-0c8d7e10a8c6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:51.019342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-3dc89b66-82d1-4b79-8e88-0c8d7e10a8c6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:45:51.019342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:51.019342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:51.019342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:51.019342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:51.019342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:51.019342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:51.019342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:51.019342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:51.019342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:51.019342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:51.019342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:51.019342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:45:51.019342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:45:51.019342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:45:51.019342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:45:51.020528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:51.020528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:51.020528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:45:51.020528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:45:51.020528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:45:51.020528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:45:51.020528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:51.020528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:51.020528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:51.020528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:51.020528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:51.020528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:51.020528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:51.020528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:51.020528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:51.020528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:51.020528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:51.020528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:51.021667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:51.021667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:51.021667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:45:51.021667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:45:51.021667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:45:51.021667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:45:51.021667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:45:51.021667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:45:51.021667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:45:51.021667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:45:51.021667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 634/1253] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:45:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:51.028843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467988c8-021b-41fa-85de-b35ea46375fa None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:51.094218 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:51.155247 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467988c8-021b-41fa-85de-b35ea46375fa None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:51.204395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c07cd-8734-49e6-be4c-e99415b8590d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:51.205037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c07cd-8734-49e6-be4c-e99415b8590d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:51.205364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c07cd-8734-49e6-be4c-e99415b8590d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:51.205669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c07cd-8734-49e6-be4c-e99415b8590d None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:51.206556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c07cd-8734-49e6-be4c-e99415b8590d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:51.206932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c07cd-8734-49e6-be4c-e99415b8590d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:51.208500 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c07cd-8734-49e6-be4c-e99415b8590d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:51.232351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 620/1254] 10.222.0.67 () {56 vars in 1143 bytes} [Wed Aug 24 08:45:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:45:51.238435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc89b66-82d1-4b79-8e88-0c8d7e10a8c6 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:51.360349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc89b66-82d1-4b79-8e88-0c8d7e10a8c6 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:51.399969 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08527e3a-ec0a-49ec-bb08-e782d09fb9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:51.400623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08527e3a-ec0a-49ec-bb08-e782d09fb9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:51.400895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08527e3a-ec0a-49ec-bb08-e782d09fb9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:51.401162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08527e3a-ec0a-49ec-bb08-e782d09fb9b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:51.470460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:51.490487 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08527e3a-ec0a-49ec-bb08-e782d09fb9b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:51.492355 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08527e3a-ec0a-49ec-bb08-e782d09fb9b8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:51.613384 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 635/1255] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:45:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 08:45:51.621093 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c07cd-8734-49e6-be4c-e99415b8590d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:51.754468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c07cd-8734-49e6-be4c-e99415b8590d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:51.768614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:51.792036 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20baa3f2-8987-47b3-85cc-f48d8ab096e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:51.792698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baa3f2-8987-47b3-85cc-f48d8ab096e6 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:51.793517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baa3f2-8987-47b3-85cc-f48d8ab096e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:51.793793 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baa3f2-8987-47b3-85cc-f48d8ab096e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:51.805265 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20baa3f2-8987-47b3-85cc-f48d8ab096e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:51.807123 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20baa3f2-8987-47b3-85cc-f48d8ab096e6 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:51.825470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-20baa3f2-8987-47b3-85cc-f48d8ab096e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:51.825470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:51.825470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:51.825470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:51.825470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:51.825470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:51.825470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:51.825470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:51.825470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:51.825470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:51.825470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:51.825470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:51.825470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:45:51.825470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:45:51.825470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:45:51.825470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:45:51.826586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:51.826586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:51.826586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:51.826586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:51.826586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:51.826586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:51.826586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:51.826586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:51.826586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:51.826586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:51.826586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:51.826586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:51.826586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:51.826586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:51.826586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:51.826586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:51.826586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:45:51.827492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:45:51.827492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:51.827492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:45:51.827492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 621/1256] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:45:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:51.832769 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08527e3a-ec0a-49ec-bb08-e782d09fb9b8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:51.931263 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:51.945672 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08527e3a-ec0a-49ec-bb08-e782d09fb9b8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:51.983523 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd4be6-ce89-4702-bc33-a7c0632f75d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:51.984162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd4be6-ce89-4702-bc33-a7c0632f75d1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:51.984443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd4be6-ce89-4702-bc33-a7c0632f75d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:51.984714 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd4be6-ce89-4702-bc33-a7c0632f75d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:51.985361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd4be6-ce89-4702-bc33-a7c0632f75d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:51.985662 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd4be6-ce89-4702-bc33-a7c0632f75d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:51.987038 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd4be6-ce89-4702-bc33-a7c0632f75d1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:52.004956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 636/1257] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:52.006095 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20baa3f2-8987-47b3-85cc-f48d8ab096e6 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:52.121065 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20baa3f2-8987-47b3-85cc-f48d8ab096e6 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:52.166104 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62eeb4-444f-478d-912b-32d71d438cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:52.166769 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62eeb4-444f-478d-912b-32d71d438cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:52.167068 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62eeb4-444f-478d-912b-32d71d438cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:52.167341 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62eeb4-444f-478d-912b-32d71d438cf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:52.176299 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62eeb4-444f-478d-912b-32d71d438cf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:52.177883 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62eeb4-444f-478d-912b-32d71d438cf7 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:52.190800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-8c62eeb4-444f-478d-912b-32d71d438cf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:45:52.190800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:52.190800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:52.190800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:52.190800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:52.190800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:52.190800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:52.190800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:52.190800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:52.190800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:52.190800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:52.190800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:52.190800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:45:52.190800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:45:52.190800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:45:52.192142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:45:52.192142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:52.192142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:52.192142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:52.192142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:52.192142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:52.192142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:52.192142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:52.192142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:52.192142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:52.192142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:52.192142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:52.192142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:52.192142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:52.192142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:52.192142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:52.192142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:52.193468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:45:52.193468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:45:52.193468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:45:52.193468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:45:52.193468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:45:52.193468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:45:52.193468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:45:52.193468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:45:52.193468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 622/1258] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:45:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:52.198400 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd4be6-ce89-4702-bc33-a7c0632f75d1 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:52.312648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd4be6-ce89-4702-bc33-a7c0632f75d1 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:52.352942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f68d1a-9f85-4d82-b595-4f01bc79dd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:52.353781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f68d1a-9f85-4d82-b595-4f01bc79dd08 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:52.354170 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f68d1a-9f85-4d82-b595-4f01bc79dd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:52.354569 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f68d1a-9f85-4d82-b595-4f01bc79dd08 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:52.355782 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f68d1a-9f85-4d82-b595-4f01bc79dd08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:52.356242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f68d1a-9f85-4d82-b595-4f01bc79dd08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:52.357796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f68d1a-9f85-4d82-b595-4f01bc79dd08 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:52.371579 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 637/1259] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:45:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:52.378348 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62eeb4-444f-478d-912b-32d71d438cf7 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:52.405934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:52.487133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62eeb4-444f-478d-912b-32d71d438cf7 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:52.524404 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ea5d2-5cfb-45c5-9066-94c01981f6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:52.525044 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ea5d2-5cfb-45c5-9066-94c01981f6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:52.525357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ea5d2-5cfb-45c5-9066-94c01981f6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:52.525654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ea5d2-5cfb-45c5-9066-94c01981f6c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:52.526481 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ea5d2-5cfb-45c5-9066-94c01981f6c8 None admin] RBAC: Adding query filter params (role.id=b13ffa72344c4856901b1ba003e273fd, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=33a0844a1c8f4ea4932e12d53e5ee9ef) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:52.526813 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ea5d2-5cfb-45c5-9066-94c01981f6c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:52.528982 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ea5d2-5cfb-45c5-9066-94c01981f6c8 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:52.536480 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 623/1260] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:45:52 2022] GET /identity/v3/role_assignments?user.id=33a0844a1c8f4ea4932e12d53e5ee9ef&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=b13ffa72344c4856901b1ba003e273fd => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:54.479792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e182fd-96a8-42e6-b2ca-042d84a1c46d None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:54.480629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e182fd-96a8-42e6-b2ca-042d84a1c46d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:54.480629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e182fd-96a8-42e6-b2ca-042d84a1c46d None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:54.481998 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 638/1261] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:45:54.487262 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d4d5b-a466-4898-8570-ea04faaa746e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:54.487827 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d4d5b-a466-4898-8570-ea04faaa746e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:54.487827 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d4d5b-a466-4898-8570-ea04faaa746e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:54.506636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-009d4d5b-a466-4898-8570-ea04faaa746e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:45:54.507827 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-009d4d5b-a466-4898-8570-ea04faaa746e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:45:54.573795 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 624/1262] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:45:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:45:54.842665 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e182fd-96a8-42e6-b2ca-042d84a1c46d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:54.853551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:54.944141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e182fd-96a8-42e6-b2ca-042d84a1c46d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:54.978778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3094698-aadb-4066-9e40-c99a899d86e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:54.979391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3094698-aadb-4066-9e40-c99a899d86e0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:54.979665 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3094698-aadb-4066-9e40-c99a899d86e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:54.979926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3094698-aadb-4066-9e40-c99a899d86e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:54.990479 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3094698-aadb-4066-9e40-c99a899d86e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:45:54.995100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d3094698-aadb-4066-9e40-c99a899d86e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:45:54.995100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:54.995100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:54.995100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:54.995100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:54.995100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:54.995100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:54.995100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:54.995100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:54.995100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:54.995100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:54.995100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:54.995100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:45:54.995100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:45:54.995100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:45:54.995100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:45:54.996327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:45:54.996327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:45:54.996327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:45:54.996327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:45:54.996327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:54.996327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:54.996327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:54.996327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:54.996327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:54.996327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:54.996327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:54.996327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:54.996327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:54.996327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:54.996327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:54.996327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:54.997289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:54.997289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:54.997289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:54.997289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:54.997289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:45:54.997289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:45:54.997289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:45:54.997289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:45:54.997289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:45:54.997289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:45:54.997289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:45:54.997289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:45:54.997289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 639/1263] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:45:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:55.002260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d4d5b-a466-4898-8570-ea04faaa746e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:55.021076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:55.101639 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d4d5b-a466-4898-8570-ea04faaa746e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:55.135301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54736df6-a78d-4df2-9450-2f583af8a617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:55.135949 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54736df6-a78d-4df2-9450-2f583af8a617 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:55.136259 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54736df6-a78d-4df2-9450-2f583af8a617 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:55.136532 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54736df6-a78d-4df2-9450-2f583af8a617 None admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:55.138402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54736df6-a78d-4df2-9450-2f583af8a617 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:55.138725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54736df6-a78d-4df2-9450-2f583af8a617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:55.140134 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54736df6-a78d-4df2-9450-2f583af8a617 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:55.148848 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 625/1264] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:45:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:55.154736 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3094698-aadb-4066-9e40-c99a899d86e0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:55.220424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:55.253977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3094698-aadb-4066-9e40-c99a899d86e0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:55.286688 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0eed0-edab-42b8-9429-8b897a899f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:55.287313 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0eed0-edab-42b8-9429-8b897a899f7d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:55.287588 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0eed0-edab-42b8-9429-8b897a899f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:55.287855 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0eed0-edab-42b8-9429-8b897a899f7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:55.298204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0eed0-edab-42b8-9429-8b897a899f7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:55.299629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0eed0-edab-42b8-9429-8b897a899f7d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:55.314422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-36e0eed0-edab-42b8-9429-8b897a899f7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:55.314422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:55.314422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:55.314422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:55.314422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:55.314422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:55.314422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:55.314422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:55.314422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:55.314422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:55.314422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:55.314422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:55.314422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:45:55.314422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:45:55.314422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:45:55.314422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:45:55.315554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:55.315554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:55.315554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:55.315554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:55.315554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:55.315554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:55.315554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:55.315554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:55.315554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:55.315554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:55.315554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:55.315554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:55.315554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:55.315554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:55.315554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:55.315554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:55.315554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:45:55.316887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:45:55.316887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:55.316887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:45:55.316887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 640/1265] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:45:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:55.321354 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54736df6-a78d-4df2-9450-2f583af8a617 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:55.402120 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:55.423534 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54736df6-a78d-4df2-9450-2f583af8a617 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:55.463131 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9408268-fca8-4fc3-b3c1-dc4fadfcffea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:55.463740 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9408268-fca8-4fc3-b3c1-dc4fadfcffea None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:55.464018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9408268-fca8-4fc3-b3c1-dc4fadfcffea None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:55.464317 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9408268-fca8-4fc3-b3c1-dc4fadfcffea None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:55.465488 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9408268-fca8-4fc3-b3c1-dc4fadfcffea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:55.465789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9408268-fca8-4fc3-b3c1-dc4fadfcffea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:55.467156 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9408268-fca8-4fc3-b3c1-dc4fadfcffea None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:55.479057 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 626/1266] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:55.484958 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0eed0-edab-42b8-9429-8b897a899f7d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:55.584384 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0eed0-edab-42b8-9429-8b897a899f7d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:55.604755 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:55.620659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf6ada-156b-4937-98e2-54dd74b025f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:55.621523 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf6ada-156b-4937-98e2-54dd74b025f4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:55.621815 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf6ada-156b-4937-98e2-54dd74b025f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:55.622114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf6ada-156b-4937-98e2-54dd74b025f4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:55.630886 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf6ada-156b-4937-98e2-54dd74b025f4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:55.632364 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf6ada-156b-4937-98e2-54dd74b025f4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:55.645288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-87cf6ada-156b-4937-98e2-54dd74b025f4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:45:55.645288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:55.645288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:55.645288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:55.645288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:55.645288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:55.645288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:55.645288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:55.645288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:55.645288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:55.645288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:55.645288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:55.645288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:45:55.645288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:45:55.645288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:45:55.645288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:45:55.646659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:55.646659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:55.646659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:45:55.646659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:45:55.646659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:45:55.646659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:45:55.646659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:55.646659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:55.646659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:55.646659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:55.646659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:55.646659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:55.646659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:55.646659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:55.646659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:55.646659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:55.646659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:55.646659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:55.647851 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:55.647851 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:55.647851 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:45:55.647851 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:45:55.647851 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:45:55.647851 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:45:55.647851 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:45:55.647851 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:45:55.647851 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:45:55.647851 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:45:55.647851 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 641/1267] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:45:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:55.652515 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9408268-fca8-4fc3-b3c1-dc4fadfcffea None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:55.751750 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9408268-fca8-4fc3-b3c1-dc4fadfcffea None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:55.790352 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7bbfc-c33f-49d0-b052-1d7c7b9d3f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:55.790951 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7bbfc-c33f-49d0-b052-1d7c7b9d3f91 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:55.791231 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7bbfc-c33f-49d0-b052-1d7c7b9d3f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:55.791501 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7bbfc-c33f-49d0-b052-1d7c7b9d3f91 None admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:55.792263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7bbfc-c33f-49d0-b052-1d7c7b9d3f91 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:55.792569 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7bbfc-c33f-49d0-b052-1d7c7b9d3f91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:55.794204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7bbfc-c33f-49d0-b052-1d7c7b9d3f91 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:55.802958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:55.813072 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 627/1268] 10.222.0.67 () {56 vars in 1143 bytes} [Wed Aug 24 08:45:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:45:55.818736 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf6ada-156b-4937-98e2-54dd74b025f4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:55.918499 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf6ada-156b-4937-98e2-54dd74b025f4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:55.952616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6507b1-de98-452c-afca-c9e364a5566f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:55.953243 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6507b1-de98-452c-afca-c9e364a5566f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:55.953527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6507b1-de98-452c-afca-c9e364a5566f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:55.953800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6507b1-de98-452c-afca-c9e364a5566f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:55.964121 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6507b1-de98-452c-afca-c9e364a5566f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:55.966408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6507b1-de98-452c-afca-c9e364a5566f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:55.980180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-5a6507b1-de98-452c-afca-c9e364a5566f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:55.980180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:55.980180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:55.980180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:55.980180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:55.980180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:55.980180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:55.980180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:55.980180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:55.980180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:55.980180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:55.980180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:55.980180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:45:55.980180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:45:55.980180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:45:55.980180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:45:55.981425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:55.981425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:55.981425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:55.981425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:55.981425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:55.981425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:55.981425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:55.981425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:55.981425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:55.981425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:55.981425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:55.981425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:55.981425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:55.981425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:55.981425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:55.981425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:55.981425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:45:55.982485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:45:55.982485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:55.982485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:45:55.982485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 642/1269] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:45:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:55.990634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7bbfc-c33f-49d0-b052-1d7c7b9d3f91 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:56.091709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7bbfc-c33f-49d0-b052-1d7c7b9d3f91 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:56.126006 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556148d-627e-4433-8f07-eae473415507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:56.126612 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556148d-627e-4433-8f07-eae473415507 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:56.126895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556148d-627e-4433-8f07-eae473415507 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:56.127161 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556148d-627e-4433-8f07-eae473415507 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:56.127787 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556148d-627e-4433-8f07-eae473415507 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:56.128117 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556148d-627e-4433-8f07-eae473415507 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:56.129470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556148d-627e-4433-8f07-eae473415507 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:56.141692 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 628/1270] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:56.147956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6507b1-de98-452c-afca-c9e364a5566f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:56.180238 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:56.250279 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6507b1-de98-452c-afca-c9e364a5566f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:56.283254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6ad55-5408-4a7f-a660-f70d5d9937da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:56.283878 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6ad55-5408-4a7f-a660-f70d5d9937da None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:56.284189 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6ad55-5408-4a7f-a660-f70d5d9937da None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:56.284463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6ad55-5408-4a7f-a660-f70d5d9937da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:56.291010 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6ad55-5408-4a7f-a660-f70d5d9937da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:56.292474 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6ad55-5408-4a7f-a660-f70d5d9937da None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:56.305971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-97f6ad55-5408-4a7f-a660-f70d5d9937da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:45:56.305971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:56.305971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:56.305971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:56.305971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:56.305971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:56.305971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:56.305971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:56.305971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:56.305971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:56.305971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:56.305971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:56.305971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:45:56.305971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:45:56.305971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:45:56.307061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:45:56.307061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:56.307061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:56.307061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:56.307061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:56.307061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:56.307061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:56.307061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:56.307061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:56.307061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:56.307061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:56.307061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:56.307061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:56.307061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:56.307061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:56.307061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:56.307061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:56.308391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:45:56.308391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:45:56.308391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:45:56.308391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:45:56.308391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:45:56.308391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:45:56.308391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:45:56.308391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:45:56.308391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 643/1271] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:45:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:56.314159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556148d-627e-4433-8f07-eae473415507 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:56.373755 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:56.415823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556148d-627e-4433-8f07-eae473415507 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:56.451132 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8ab1d-6068-4ae1-a980-d9f869d3f88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:56.451636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8ab1d-6068-4ae1-a980-d9f869d3f88f None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:56.451909 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8ab1d-6068-4ae1-a980-d9f869d3f88f None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:56.452208 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8ab1d-6068-4ae1-a980-d9f869d3f88f None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:56.452893 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb8ab1d-6068-4ae1-a980-d9f869d3f88f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:56.453193 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb8ab1d-6068-4ae1-a980-d9f869d3f88f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:56.454612 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb8ab1d-6068-4ae1-a980-d9f869d3f88f None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:56.467155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 629/1272] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:45:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:56.473179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6ad55-5408-4a7f-a660-f70d5d9937da None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:56.564998 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:56.573920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6ad55-5408-4a7f-a660-f70d5d9937da None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:56.607463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddbe98-c8cc-4f66-bac3-42e15cf28d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:56.608138 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddbe98-c8cc-4f66-bac3-42e15cf28d5f None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:56.608414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddbe98-c8cc-4f66-bac3-42e15cf28d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:56.608683 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddbe98-c8cc-4f66-bac3-42e15cf28d5f None admin] PATH_INFO: `/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/33a0844a1c8f4ea4932e12d53e5ee9ef/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:56.610529 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddbe98-c8cc-4f66-bac3-42e15cf28d5f None admin] RBAC: Authorizing `identity:create_grant(project_id=fce6ee19f58b4a468c9a5d169e29a018, user_id=33a0844a1c8f4ea4932e12d53e5ee9ef, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:56.612263 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddbe98-c8cc-4f66-bac3-42e15cf28d5f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:56.629108 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 644/1273] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 24 08:45:56 2022] PUT /identity/v3/projects/fce6ee19f58b4a468c9a5d169e29a018/users/33a0844a1c8f4ea4932e12d53e5ee9ef/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:45:58.539172 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be9c52-1359-4978-bf75-36399b565200 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:58.539746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be9c52-1359-4978-bf75-36399b565200 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:58.539746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be9c52-1359-4978-bf75-36399b565200 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:58.541097 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 630/1274] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:45:58.547071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7ead2-3d1b-4605-97bc-3612ae308781 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:58.547501 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7ead2-3d1b-4605-97bc-3612ae308781 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:58.547591 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7ead2-3d1b-4605-97bc-3612ae308781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:58.566374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a5b7ead2-3d1b-4605-97bc-3612ae308781 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:45:58.567561 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a5b7ead2-3d1b-4605-97bc-3612ae308781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:45:58.638821 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 645/1275] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:45:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:45:58.914913 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be9c52-1359-4978-bf75-36399b565200 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:59.017626 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be9c52-1359-4978-bf75-36399b565200 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:59.036452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:59.053945 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759ff22-3512-4b18-bd6e-e35611003f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:59.054581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759ff22-3512-4b18-bd6e-e35611003f37 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:59.054858 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759ff22-3512-4b18-bd6e-e35611003f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:59.055126 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759ff22-3512-4b18-bd6e-e35611003f37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:59.065614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759ff22-3512-4b18-bd6e-e35611003f37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 08:45:59.070263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-6759ff22-3512-4b18-bd6e-e35611003f37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:45:59.070263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:59.070263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:59.070263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:59.070263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:59.070263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:59.070263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:59.070263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:59.070263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:59.070263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:59.070263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:59.070263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:59.070263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 08:45:59.070263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 08:45:59.070263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 08:45:59.070263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 08:45:59.071372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 08:45:59.071372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 08:45:59.071372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 08:45:59.071372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 08:45:59.071372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:59.071372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:59.071372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:59.071372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:59.071372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:59.071372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:59.071372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:59.071372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:59.071372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:59.071372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:59.071372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:59.071372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:59.072624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:59.072624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:59.072624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:59.072624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:59.072624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 08:45:59.072624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 08:45:59.072624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 08:45:59.072624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 08:45:59.072624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 08:45:59.072624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 08:45:59.072624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 08:45:59.072624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:45:59.072624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 631/1276] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 24 08:45:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:59.077526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7ead2-3d1b-4605-97bc-3612ae308781 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:59.159077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:59.177921 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7ead2-3d1b-4605-97bc-3612ae308781 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:59.212278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f02c95-6840-4e16-b184-c12b946a541f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:59.212879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f02c95-6840-4e16-b184-c12b946a541f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:59.213157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f02c95-6840-4e16-b184-c12b946a541f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:59.213427 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f02c95-6840-4e16-b184-c12b946a541f None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:59.214089 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f02c95-6840-4e16-b184-c12b946a541f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:59.214390 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f02c95-6840-4e16-b184-c12b946a541f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:59.215799 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f02c95-6840-4e16-b184-c12b946a541f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:59.223868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 646/1277] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 24 08:45:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:59.229917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759ff22-3512-4b18-bd6e-e35611003f37 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:59.331399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759ff22-3512-4b18-bd6e-e35611003f37 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:59.365668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3f6f1-192b-40e1-9380-b1515a9e86d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:59.366304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3f6f1-192b-40e1-9380-b1515a9e86d4 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:59.366582 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3f6f1-192b-40e1-9380-b1515a9e86d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:59.366851 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3f6f1-192b-40e1-9380-b1515a9e86d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:59.378093 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3f6f1-192b-40e1-9380-b1515a9e86d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:59.379549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3f6f1-192b-40e1-9380-b1515a9e86d4 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:59.393814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-35e3f6f1-192b-40e1-9380-b1515a9e86d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:59.393814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:59.393814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:59.393814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:59.393814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:59.393814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:59.393814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:59.393814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:59.393814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:59.393814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:59.393814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:59.393814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:59.393814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:45:59.393814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:45:59.393814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:45:59.393814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:45:59.395046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:59.395046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:59.395046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:59.395046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:59.395046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:59.395046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:59.395046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:59.395046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:59.395046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:59.395046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:59.395046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:59.395046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:59.395046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:59.395046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:59.395046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:59.395046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:59.395046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:45:59.395956 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:45:59.395956 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:45:59.395956 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:45:59.395956 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 632/1278] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:45:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:59.401088 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f02c95-6840-4e16-b184-c12b946a541f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:59.501015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f02c95-6840-4e16-b184-c12b946a541f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:59.533205 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c2e0a-b399-45ed-b89e-c0ef74a02acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:59.533805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c2e0a-b399-45ed-b89e-c0ef74a02acd None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:59.534080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c2e0a-b399-45ed-b89e-c0ef74a02acd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:59.534346 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c2e0a-b399-45ed-b89e-c0ef74a02acd None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:59.534983 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c2e0a-b399-45ed-b89e-c0ef74a02acd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:59.535277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c2e0a-b399-45ed-b89e-c0ef74a02acd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:59.536667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c2e0a-b399-45ed-b89e-c0ef74a02acd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:59.549223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 647/1279] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:59.555099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3f6f1-192b-40e1-9380-b1515a9e86d4 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:59.587039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:59.665148 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3f6f1-192b-40e1-9380-b1515a9e86d4 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:59.699727 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a67da-eb3a-422c-90da-daf0a386ab10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:59.700398 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a67da-eb3a-422c-90da-daf0a386ab10 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:59.700679 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a67da-eb3a-422c-90da-daf0a386ab10 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:59.700951 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a67da-eb3a-422c-90da-daf0a386ab10 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:59.708498 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6a67da-eb3a-422c-90da-daf0a386ab10 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:59.709944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6a67da-eb3a-422c-90da-daf0a386ab10 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:59.721643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-cb6a67da-eb3a-422c-90da-daf0a386ab10 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:45:59.721643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:45:59.721643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:45:59.721643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:45:59.721643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:45:59.721643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:45:59.721643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:45:59.721643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:45:59.721643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:45:59.721643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:45:59.721643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:45:59.721643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:45:59.721643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 08:45:59.721643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 08:45:59.721643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 08:45:59.721643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 08:45:59.722822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:45:59.722822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:45:59.722822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 08:45:59.722822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:45:59.722822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 08:45:59.722822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 08:45:59.722822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:45:59.722822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:45:59.722822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:45:59.722822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:45:59.722822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:45:59.722822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:45:59.722822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:45:59.722822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:45:59.722822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:45:59.722822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:45:59.722822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:45:59.722822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:45:59.724227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:45:59.724227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:45:59.724227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 08:45:59.724227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 08:45:59.724227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 08:45:59.724227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 08:45:59.724227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 08:45:59.724227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 08:45:59.724227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 08:45:59.724227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:45:59.724227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 633/1280] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:45:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:45:59.728589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c2e0a-b399-45ed-b89e-c0ef74a02acd None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:59.732986 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:59.830011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c2e0a-b399-45ed-b89e-c0ef74a02acd None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:45:59.866141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fcbba-209f-42b7-be57-df84d5af1c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:45:59.866751 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fcbba-209f-42b7-be57-df84d5af1c2a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:45:59.867030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fcbba-209f-42b7-be57-df84d5af1c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:45:59.867308 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fcbba-209f-42b7-be57-df84d5af1c2a None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:45:59.868041 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1fcbba-209f-42b7-be57-df84d5af1c2a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:45:59.868383 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1fcbba-209f-42b7-be57-df84d5af1c2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:45:59.869808 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1fcbba-209f-42b7-be57-df84d5af1c2a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:45:59.888117 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 648/1281] 10.222.0.67 () {56 vars in 1143 bytes} [Wed Aug 24 08:45:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:45:59.893880 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a67da-eb3a-422c-90da-daf0a386ab10 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:45:59.986835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:45:59.997254 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a67da-eb3a-422c-90da-daf0a386ab10 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:46:00.031129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a47640-7b14-4341-abc6-654e365803b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:46:00.031728 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a47640-7b14-4341-abc6-654e365803b3 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:00.032005 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a47640-7b14-4341-abc6-654e365803b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:00.032303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a47640-7b14-4341-abc6-654e365803b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:00.099594 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a47640-7b14-4341-abc6-654e365803b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:46:00.100979 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a47640-7b14-4341-abc6-654e365803b3 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:46:00.189867 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:46:00.200782 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 634/1282] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 24 08:45:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 08:46:00.206599 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fcbba-209f-42b7-be57-df84d5af1c2a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:46:00.268327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:46:00.308341 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fcbba-209f-42b7-be57-df84d5af1c2a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:46:00.343620 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cab0d5-f9c3-428e-8fca-b8023f548eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:46:00.344280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cab0d5-f9c3-428e-8fca-b8023f548eaa None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:00.344559 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cab0d5-f9c3-428e-8fca-b8023f548eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:00.344830 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cab0d5-f9c3-428e-8fca-b8023f548eaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:00.354956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cab0d5-f9c3-428e-8fca-b8023f548eaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:46:00.356482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cab0d5-f9c3-428e-8fca-b8023f548eaa None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:46:00.370942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-37cab0d5-f9c3-428e-8fca-b8023f548eaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:46:00.370942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:46:00.370942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:46:00.370942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:46:00.370942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:46:00.370942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:46:00.370942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:46:00.370942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:46:00.370942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:46:00.370942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:46:00.370942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:46:00.370942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:46:00.370942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 08:46:00.370942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 08:46:00.370942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 08:46:00.370942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 08:46:00.372885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:46:00.372885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:46:00.372885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:46:00.372885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:46:00.372885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:46:00.372885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:46:00.372885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:46:00.372885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:46:00.372885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:46:00.372885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:46:00.372885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:46:00.372885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:46:00.372885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:46:00.372885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:46:00.372885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:46:00.372885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:46:00.372885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 08:46:00.373807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 08:46:00.373807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 08:46:00.373807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:46:00.373807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 649/1283] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:46:00.378684 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a47640-7b14-4341-abc6-654e365803b3 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:46:00.479068 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a47640-7b14-4341-abc6-654e365803b3 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:46:00.515711 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22cadf-3ad0-496d-9b06-f7e9bdbecb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:46:00.516346 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22cadf-3ad0-496d-9b06-f7e9bdbecb42 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:00.516622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22cadf-3ad0-496d-9b06-f7e9bdbecb42 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:00.516889 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22cadf-3ad0-496d-9b06-f7e9bdbecb42 None admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:00.517528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22cadf-3ad0-496d-9b06-f7e9bdbecb42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:46:00.517824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22cadf-3ad0-496d-9b06-f7e9bdbecb42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:46:00.519177 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22cadf-3ad0-496d-9b06-f7e9bdbecb42 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:46:00.531270 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 635/1284] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 24 08:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:46:00.537026 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cab0d5-f9c3-428e-8fca-b8023f548eaa None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:46:00.641733 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cab0d5-f9c3-428e-8fca-b8023f548eaa None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:46:00.656031 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:46:00.676744 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dab335-2425-49b6-b2b0-091d82c8177e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:46:00.677382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dab335-2425-49b6-b2b0-091d82c8177e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:00.677657 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dab335-2425-49b6-b2b0-091d82c8177e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:00.677926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dab335-2425-49b6-b2b0-091d82c8177e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:00.684473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dab335-2425-49b6-b2b0-091d82c8177e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:46:00.685914 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dab335-2425-49b6-b2b0-091d82c8177e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:46:00.697780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-78dab335-2425-49b6-b2b0-091d82c8177e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:46:00.697780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:46:00.697780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:46:00.697780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:46:00.697780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:46:00.697780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:46:00.697780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:46:00.697780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:46:00.697780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:46:00.697780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:46:00.697780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:46:00.697780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:46:00.697780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 08:46:00.697780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 08:46:00.697780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 08:46:00.698876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 08:46:00.698876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:46:00.698876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:46:00.698876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:46:00.698876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:46:00.698876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:46:00.698876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:46:00.698876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:46:00.698876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Aug 24 08:46:00.698876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:46:00.698876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:46:00.698876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:46:00.698876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:46:00.698876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:46:00.698876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:46:00.698876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:46:00.698876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:46:00.700526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 08:46:00.700526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 08:46:00.700526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 08:46:00.700526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 08:46:00.700526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 08:46:00.700526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 08:46:00.700526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 08:46:00.700526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Aug 24 08:46:00.700526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 650/1285] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 24 08:46:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:46:00.705581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22cadf-3ad0-496d-9b06-f7e9bdbecb42 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:46:00.757380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:46:00.813549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22cadf-3ad0-496d-9b06-f7e9bdbecb42 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:46:00.847875 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d8f36-34ad-419b-956b-b00d59e6b0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:46:00.848507 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d8f36-34ad-419b-956b-b00d59e6b0bf None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:00.848782 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d8f36-34ad-419b-956b-b00d59e6b0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:00.849046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d8f36-34ad-419b-956b-b00d59e6b0bf None admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:00.849730 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d8f36-34ad-419b-956b-b00d59e6b0bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:46:00.850030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d8f36-34ad-419b-956b-b00d59e6b0bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:46:00.851432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d8f36-34ad-419b-956b-b00d59e6b0bf None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:46:00.863999 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 636/1286] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 24 08:46:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:46:00.870110 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dab335-2425-49b6-b2b0-091d82c8177e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:46:00.972205 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dab335-2425-49b6-b2b0-091d82c8177e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:46:01.005694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899dce1-b816-430b-8268-bca775b75b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:46:01.006290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899dce1-b816-430b-8268-bca775b75b95 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:01.006563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899dce1-b816-430b-8268-bca775b75b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:01.006827 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899dce1-b816-430b-8268-bca775b75b95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:01.007544 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899dce1-b816-430b-8268-bca775b75b95 None admin] RBAC: Adding query filter params (role.id=b13ffa72344c4856901b1ba003e273fd, scope.project.id=fce6ee19f58b4a468c9a5d169e29a018, user.id=33a0844a1c8f4ea4932e12d53e5ee9ef) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:46:01.007847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899dce1-b816-430b-8268-bca775b75b95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:46:01.009295 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899dce1-b816-430b-8268-bca775b75b95 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:46:01.016186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 651/1287] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 24 08:46:00 2022] GET /identity/v3/role_assignments?user.id=33a0844a1c8f4ea4932e12d53e5ee9ef&scope.project.id=fce6ee19f58b4a468c9a5d169e29a018&role.id=b13ffa72344c4856901b1ba003e273fd => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:46:02.993305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd256467-e13f-4e71-8cf5-e04428ebc5c8 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:02.993863 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd256467-e13f-4e71-8cf5-e04428ebc5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:02.993863 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd256467-e13f-4e71-8cf5-e04428ebc5c8 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:02.995088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 637/1288] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:46:03.001008 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775ceea-27f9-4973-a34b-ffb2ad99b09e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:03.001446 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775ceea-27f9-4973-a34b-ffb2ad99b09e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:03.001545 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775ceea-27f9-4973-a34b-ffb2ad99b09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:03.019400 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f775ceea-27f9-4973-a34b-ffb2ad99b09e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:46:03.020610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f775ceea-27f9-4973-a34b-ffb2ad99b09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:46:03.050599 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:46:03.090029 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 652/1289] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:46:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:46:03.355382 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd256467-e13f-4e71-8cf5-e04428ebc5c8 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:46:03.458453 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd256467-e13f-4e71-8cf5-e04428ebc5c8 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:46:03.469219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:46:03.492699 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa4846-9139-4435-bcad-1f27bf785a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:46:03.493327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa4846-9139-4435-bcad-1f27bf785a15 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:03.493608 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa4846-9139-4435-bcad-1f27bf785a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:03.493878 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa4846-9139-4435-bcad-1f27bf785a15 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:03.494462 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa4846-9139-4435-bcad-1f27bf785a15 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:46:03.495816 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa4846-9139-4435-bcad-1f27bf785a15 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:46:03.504920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-68aa4846-9139-4435-bcad-1f27bf785a15 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:46:03.504920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:46:03.504920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:46:03.504920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:46:03.504920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:46:03.504920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:46:03.504920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:46:03.504920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:46:03.504920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:46:03.504920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:46:03.504920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:46:03.504920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:46:03.504920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:46:03.504920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:46:03.504920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:46:03.504920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:46:03.504920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:46:03.508363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:46:03.508363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:46:03.508363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:46:03.508363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:46:03.508363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:46:03.508363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:46:03.508363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:46:03.508363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:46:03.508363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:46:03.508363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:46:03.508363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:46:03.508363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:46:03.508363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:46:03.508363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:46:03.508363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:46:03.508363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:46:03.509492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:46:03.509492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:46:03.509492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:46:03.509492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:46:03.509492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:46:03.509492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:46:03.509492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:46:03.509492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:46:03.509492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:46:03.509492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:46:03.509492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:46:03.509492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:46:03.509492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:46:03.509492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:46:03.509492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 638/1290] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 24 08:46:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:46:03.512075 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775ceea-27f9-4973-a34b-ffb2ad99b09e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:46:03.608333 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775ceea-27f9-4973-a34b-ffb2ad99b09e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:46:03.653462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc48db-5e8e-45e7-8aa6-e09247550677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:46:03.654164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc48db-5e8e-45e7-8aa6-e09247550677 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:03.654453 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc48db-5e8e-45e7-8aa6-e09247550677 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:03.654771 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc48db-5e8e-45e7-8aa6-e09247550677 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:03.655463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc48db-5e8e-45e7-8aa6-e09247550677 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:46:03.655837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc48db-5e8e-45e7-8aa6-e09247550677 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:46:03.657336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc48db-5e8e-45e7-8aa6-e09247550677 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:46:03.663127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 653/1291] 10.222.0.67 () {56 vars in 1116 bytes} [Wed Aug 24 08:46:03 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:46:03.669208 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa4846-9139-4435-bcad-1f27bf785a15 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:46:03.775796 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa4846-9139-4435-bcad-1f27bf785a15 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:46:03.808905 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee558de7-d6be-417a-8fe4-1dd1d41176d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:46:03.810023 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee558de7-d6be-417a-8fe4-1dd1d41176d1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:03.810302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee558de7-d6be-417a-8fe4-1dd1d41176d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:03.810570 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee558de7-d6be-417a-8fe4-1dd1d41176d1 None admin] PATH_INFO: `/v3/services` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:03.811196 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee558de7-d6be-417a-8fe4-1dd1d41176d1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:46:03.811499 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee558de7-d6be-417a-8fe4-1dd1d41176d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:46:03.813170 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee558de7-d6be-417a-8fe4-1dd1d41176d1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:46:03.817645 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 639/1292] 10.222.0.67 () {56 vars in 1116 bytes} [Wed Aug 24 08:46:03 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:46:05.719896 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9951e9b-2922-45bc-99de-6b34075565d8 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:05.720669 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9951e9b-2922-45bc-99de-6b34075565d8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:05.720669 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9951e9b-2922-45bc-99de-6b34075565d8 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:05.721852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 654/1293] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:46:05.727450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7541777-6e81-4a51-ae7c-0e2ed6f72790 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:05.727919 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7541777-6e81-4a51-ae7c-0e2ed6f72790 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:05.728020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7541777-6e81-4a51-ae7c-0e2ed6f72790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:05.746179 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-c7541777-6e81-4a51-ae7c-0e2ed6f72790 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:46:05.747917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c7541777-6e81-4a51-ae7c-0e2ed6f72790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:46:05.777184 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:46:05.815230 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 640/1294] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:46:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:46:06.081926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9951e9b-2922-45bc-99de-6b34075565d8 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:46:06.114682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:46:06.188788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9951e9b-2922-45bc-99de-6b34075565d8 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:46:06.224551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752b92a-0789-441d-90c5-bf4a35819d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:46:06.225161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752b92a-0789-441d-90c5-bf4a35819d3a None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:06.225439 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752b92a-0789-441d-90c5-bf4a35819d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:06.225705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752b92a-0789-441d-90c5-bf4a35819d3a None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:06.226362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752b92a-0789-441d-90c5-bf4a35819d3a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:46:06.227711 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752b92a-0789-441d-90c5-bf4a35819d3a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:46:06.242910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 655/1295] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 24 08:46:06 2022] POST /identity/v3/services => generated 246 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:46:08.197238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0c6d8-a6b7-454c-8471-a668431966eb None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:08.197780 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0c6d8-a6b7-454c-8471-a668431966eb None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:08.197780 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0c6d8-a6b7-454c-8471-a668431966eb None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:08.199142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 641/1296] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:46:08.204699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcd902-4e2f-431a-a749-b622c5df123b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:08.205151 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcd902-4e2f-431a-a749-b622c5df123b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:08.205250 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcd902-4e2f-431a-a749-b622c5df123b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:08.223303 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-11fcd902-4e2f-431a-a749-b622c5df123b None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:46:08.224505 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-11fcd902-4e2f-431a-a749-b622c5df123b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:46:08.296622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 656/1297] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:46:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:46:08.571961 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0c6d8-a6b7-454c-8471-a668431966eb None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:46:08.692318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0c6d8-a6b7-454c-8471-a668431966eb None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:46:08.728379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b819b-90db-4883-8b88-6f3c4090ca7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:46:08.729014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b819b-90db-4883-8b88-6f3c4090ca7d None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:08.729293 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b819b-90db-4883-8b88-6f3c4090ca7d None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:08.729563 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b819b-90db-4883-8b88-6f3c4090ca7d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:08.730149 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b819b-90db-4883-8b88-6f3c4090ca7d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:46:08.732451 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b819b-90db-4883-8b88-6f3c4090ca7d None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:46:08.741305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-bc9b819b-90db-4883-8b88-6f3c4090ca7d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:46:08.741305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:46:08.741305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:46:08.741305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:46:08.741305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:46:08.741305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:46:08.741305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:46:08.741305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:46:08.741305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:46:08.741305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:46:08.741305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:46:08.741305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:46:08.741305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:46:08.741305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:46:08.741305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:46:08.741305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:46:08.741305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:46:08.742450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:46:08.742450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:46:08.742450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:46:08.742450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:46:08.742450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:46:08.742450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:46:08.742450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:46:08.742450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:46:08.742450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:46:08.742450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:46:08.742450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:46:08.742450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:46:08.742450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:46:08.742450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:46:08.742450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:46:08.742450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:46:08.743492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:46:08.743492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:46:08.743492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:46:08.743492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:46:08.743492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:46:08.743492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:46:08.743492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:46:08.743492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:46:08.743492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:46:08.743492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:46:08.743492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:46:08.743492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:46:08.743492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:46:08.743492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:46:08.743492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 642/1298] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 24 08:46:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:46:08.748239 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcd902-4e2f-431a-a749-b622c5df123b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:46:08.767603 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:46:08.850998 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcd902-4e2f-431a-a749-b622c5df123b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:46:08.885908 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469227f-5cc4-4875-a97c-61a7340b9535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:46:08.886513 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469227f-5cc4-4875-a97c-61a7340b9535 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:08.886791 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469227f-5cc4-4875-a97c-61a7340b9535 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:08.887062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469227f-5cc4-4875-a97c-61a7340b9535 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:08.887783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469227f-5cc4-4875-a97c-61a7340b9535 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:46:08.888081 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469227f-5cc4-4875-a97c-61a7340b9535 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:46:08.889476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469227f-5cc4-4875-a97c-61a7340b9535 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:46:08.894512 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 657/1299] 10.222.0.67 () {56 vars in 1116 bytes} [Wed Aug 24 08:46:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:46:08.900557 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b819b-90db-4883-8b88-6f3c4090ca7d None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:46:08.932811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:46:09.003193 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b819b-90db-4883-8b88-6f3c4090ca7d None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:46:09.037239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69de503-13c4-4b95-8c11-2343602fc2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:46:09.037844 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69de503-13c4-4b95-8c11-2343602fc2e1 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:09.038121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69de503-13c4-4b95-8c11-2343602fc2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:09.038392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69de503-13c4-4b95-8c11-2343602fc2e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:09.039091 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69de503-13c4-4b95-8c11-2343602fc2e1 None admin] RBAC: Adding query filter params (interface=public, service_id=09f46311551143ffb6de4f56682acb65, region_id=RegionOne) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:46:09.039393 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69de503-13c4-4b95-8c11-2343602fc2e1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:46:09.040787 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69de503-13c4-4b95-8c11-2343602fc2e1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:46:09.046078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 643/1300] 10.222.0.67 () {56 vars in 1251 bytes} [Wed Aug 24 08:46:08 2022] GET /identity/v3/endpoints?service_id=09f46311551143ffb6de4f56682acb65&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:46:09.051727 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469227f-5cc4-4875-a97c-61a7340b9535 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:46:09.133218 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:46:09.152053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469227f-5cc4-4875-a97c-61a7340b9535 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:46:09.186200 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbf667-23f9-4417-8972-7c37f0cd5749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:46:09.186795 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbf667-23f9-4417-8972-7c37f0cd5749 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:09.187072 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbf667-23f9-4417-8972-7c37f0cd5749 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:09.187341 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbf667-23f9-4417-8972-7c37f0cd5749 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:09.188227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbf667-23f9-4417-8972-7c37f0cd5749 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:46:09.189578 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbf667-23f9-4417-8972-7c37f0cd5749 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:46:09.196246 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 658/1301] 10.222.0.67 () {56 vars in 1087 bytes} [Wed Aug 24 08:46:09 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:46:11.141541 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee576f-5f8d-4702-92ef-0b9302b594de None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:11.142100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee576f-5f8d-4702-92ef-0b9302b594de None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:11.142100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee576f-5f8d-4702-92ef-0b9302b594de None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:11.143446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 644/1302] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:46:11.149210 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e049ab-514a-4ee3-92d3-f55c75eaf378 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:11.149668 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e049ab-514a-4ee3-92d3-f55c75eaf378 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:11.149800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e049ab-514a-4ee3-92d3-f55c75eaf378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:11.168352 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-85e049ab-514a-4ee3-92d3-f55c75eaf378 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:46:11.169824 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-85e049ab-514a-4ee3-92d3-f55c75eaf378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:46:11.236376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 659/1303] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:46:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 08:46:11.504649 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee576f-5f8d-4702-92ef-0b9302b594de None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:46:11.598600 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:46:11.605883 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee576f-5f8d-4702-92ef-0b9302b594de None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:46:11.647386 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02328a8e-66cf-4373-85c1-17f366e441f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:46:11.648017 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02328a8e-66cf-4373-85c1-17f366e441f0 None admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:11.648368 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02328a8e-66cf-4373-85c1-17f366e441f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:11.648644 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02328a8e-66cf-4373-85c1-17f366e441f0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:11.649254 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02328a8e-66cf-4373-85c1-17f366e441f0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:46:11.650600 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02328a8e-66cf-4373-85c1-17f366e441f0 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:46:11.659391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application [None req-02328a8e-66cf-4373-85c1-17f366e441f0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:46:11.659391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:46:11.659391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 08:46:11.659391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 08:46:11.659391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 08:46:11.659391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 08:46:11.659391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 08:46:11.659391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:46:11.659391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:46:11.659391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:46:11.659391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 08:46:11.659391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:46:11.659391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 08:46:11.659391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 08:46:11.659391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 08:46:11.659391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 08:46:11.659391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 08:46:11.660577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 08:46:11.660577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 08:46:11.660577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 08:46:11.660577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 08:46:11.660577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 08:46:11.660577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 08:46:11.660577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 08:46:11.660577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 08:46:11.660577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 08:46:11.660577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 08:46:11.660577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 08:46:11.660577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 08:46:11.660577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 08:46:11.660577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 08:46:11.660577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 08:46:11.660577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 08:46:11.661437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 08:46:11.661437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application with Lock( Aug 24 08:46:11.661437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 08:46:11.661437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self._enter() Aug 24 08:46:11.661437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 08:46:11.661437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 08:46:11.661437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 08:46:11.661437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application return self.creator() Aug 24 08:46:11.661437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 08:46:11.661437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application created_value = creator( Aug 24 08:46:11.661437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 08:46:11.661437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 08:46:11.661437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 08:46:11.661437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: ERROR keystone.server.flask.application Aug 24 08:46:11.661437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 645/1304] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 24 08:46:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 08:46:11.666528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e049ab-514a-4ee3-92d3-f55c75eaf378 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:46:11.743115 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:46:11.768189 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e049ab-514a-4ee3-92d3-f55c75eaf378 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:46:11.802272 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f59545-c58a-4a0a-b09e-3249da50ae43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:46:11.802865 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f59545-c58a-4a0a-b09e-3249da50ae43 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:11.803139 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f59545-c58a-4a0a-b09e-3249da50ae43 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:11.803412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f59545-c58a-4a0a-b09e-3249da50ae43 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:11.804030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f59545-c58a-4a0a-b09e-3249da50ae43 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:46:11.804356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f59545-c58a-4a0a-b09e-3249da50ae43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:46:11.805712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f59545-c58a-4a0a-b09e-3249da50ae43 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:46:11.811170 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 660/1305] 10.222.0.67 () {56 vars in 1116 bytes} [Wed Aug 24 08:46:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:46:11.817357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02328a8e-66cf-4373-85c1-17f366e441f0 None admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:46:11.917026 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02328a8e-66cf-4373-85c1-17f366e441f0 None admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:46:11.955670 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f099d6-c98d-4371-a955-0030bbe384a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:46:11.956382 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f099d6-c98d-4371-a955-0030bbe384a1 None admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:46:11.956666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f099d6-c98d-4371-a955-0030bbe384a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:46:11.956941 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f099d6-c98d-4371-a955-0030bbe384a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:46:11.957611 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f099d6-c98d-4371-a955-0030bbe384a1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:46:11.958970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f099d6-c98d-4371-a955-0030bbe384a1 None admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:46:11.983498 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 646/1306] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:46:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:46:21.807720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=30 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:46:21.978140 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:18.161599 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec8199-961c-47da-89e5-ec9aa863b65d None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:18.162179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec8199-961c-47da-89e5-ec9aa863b65d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:18.162179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec8199-961c-47da-89e5-ec9aa863b65d None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:18.167504 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 661/1307] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:47:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:18.168873 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b0b66-5e7a-45ee-974f-fe9487959c02 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:18.169313 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b0b66-5e7a-45ee-974f-fe9487959c02 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:18.169423 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b0b66-5e7a-45ee-974f-fe9487959c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:18.245388 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-019b0b66-5e7a-45ee-974f-fe9487959c02 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:18.246754 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-019b0b66-5e7a-45ee-974f-fe9487959c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:18.419962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 647/1308] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:18.735665 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430b337-acff-464b-8ea1-ac7f5f03a5b8 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:18.735665 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430b337-acff-464b-8ea1-ac7f5f03a5b8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:18.735665 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430b337-acff-464b-8ea1-ac7f5f03a5b8 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:18.741691 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 662/1309] 10.222.0.67 () {54 vars in 965 bytes} [Wed Aug 24 08:47:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:18.746095 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36d0d3-48bf-4990-9ee6-7db1a70a16c9 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:18.746496 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36d0d3-48bf-4990-9ee6-7db1a70a16c9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:18.746635 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36d0d3-48bf-4990-9ee6-7db1a70a16c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:18.825883 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-fd36d0d3-48bf-4990-9ee6-7db1a70a16c9 None None] MFA Rules not processed for user `2ee2e93901914b1d82607a3ae0e2acf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:18.827079 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fd36d0d3-48bf-4990-9ee6-7db1a70a16c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:18.850265 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:18.914850 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 648/1310] 10.222.0.67 () {58 vars in 1062 bytes} [Wed Aug 24 08:47:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:18.926703 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430b337-acff-464b-8ea1-ac7f5f03a5b8 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:18.928449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7430b337-acff-464b-8ea1-ac7f5f03a5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:19.057901 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430b337-acff-464b-8ea1-ac7f5f03a5b8 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:19.142312 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d901a91-abb6-4397-af39-44757129e2b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:19.143017 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d901a91-abb6-4397-af39-44757129e2b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:19.143361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d901a91-abb6-4397-af39-44757129e2b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:19.143691 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d901a91-abb6-4397-af39-44757129e2b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:19.145402 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8d901a91-abb6-4397-af39-44757129e2b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:19.206457 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:19.261905 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d901a91-abb6-4397-af39-44757129e2b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:19.263426 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d901a91-abb6-4397-af39-44757129e2b1 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:19.283569 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 663/1311] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:21.581470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05657c82-a6a1-4ec6-9468-0175af29d007 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:21.582305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05657c82-a6a1-4ec6-9468-0175af29d007 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:21.582305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05657c82-a6a1-4ec6-9468-0175af29d007 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:21.583351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 649/1312] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:21.588820 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cd877-79c2-40e8-a365-68132f5bc64d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:21.589231 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cd877-79c2-40e8-a365-68132f5bc64d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:21.589412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cd877-79c2-40e8-a365-68132f5bc64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:21.673312 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-382cd877-79c2-40e8-a365-68132f5bc64d None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:21.674523 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-382cd877-79c2-40e8-a365-68132f5bc64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:21.780343 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 664/1313] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:22.068946 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05657c82-a6a1-4ec6-9468-0175af29d007 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:22.070462 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-05657c82-a6a1-4ec6-9468-0175af29d007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:22.176870 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05657c82-a6a1-4ec6-9468-0175af29d007 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:22.247557 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:22.250331 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fbc19-4691-4780-8c56-0bbf746e7d6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:22.251189 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fbc19-4691-4780-8c56-0bbf746e7d6a service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:22.251531 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fbc19-4691-4780-8c56-0bbf746e7d6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:22.251959 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fbc19-4691-4780-8c56-0bbf746e7d6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:22.253785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fc9fbc19-4691-4780-8c56-0bbf746e7d6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:22.366296 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9fbc19-4691-4780-8c56-0bbf746e7d6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:22.367634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9fbc19-4691-4780-8c56-0bbf746e7d6a service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:22.387524 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 650/1314] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:24.640661 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851ed96-c507-4f28-bdc5-77281762892d None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:24.642615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851ed96-c507-4f28-bdc5-77281762892d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:24.642615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851ed96-c507-4f28-bdc5-77281762892d None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:24.643540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 665/1315] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:47:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:24.648641 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2c97d-c63b-4e1e-93d6-9336645ac72d None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:24.649099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2c97d-c63b-4e1e-93d6-9336645ac72d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:24.649305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2c97d-c63b-4e1e-93d6-9336645ac72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:24.726960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-47c2c97d-c63b-4e1e-93d6-9336645ac72d None None] MFA Rules not processed for user `98c373122e5c4b1a947b0dbd458f5b95`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:24.728214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-47c2c97d-c63b-4e1e-93d6-9336645ac72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:24.779707 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:24.826775 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 651/1316] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:24 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:25.113656 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3851ed96-c507-4f28-bdc5-77281762892d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:25.115169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3851ed96-c507-4f28-bdc5-77281762892d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:25.124933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:25.222442 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3851ed96-c507-4f28-bdc5-77281762892d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:25.297494 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc661bf2-c90f-49b6-9199-f782920c6fa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:25.298114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc661bf2-c90f-49b6-9199-f782920c6fa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:25.298393 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc661bf2-c90f-49b6-9199-f782920c6fa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:25.298663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc661bf2-c90f-49b6-9199-f782920c6fa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:25.300316 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bc661bf2-c90f-49b6-9199-f782920c6fa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:25.364746 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:25.414600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc661bf2-c90f-49b6-9199-f782920c6fa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:25.416349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc661bf2-c90f-49b6-9199-f782920c6fa6 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:25.434523 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 666/1317] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:25 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:27.887372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98346dfc-4325-4656-976d-df8af56a420f None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:27.887930 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98346dfc-4325-4656-976d-df8af56a420f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:27.887930 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98346dfc-4325-4656-976d-df8af56a420f None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:27.889343 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 652/1318] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:27.895234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7e8f6-7a10-45c5-8c82-8494c3afe1d2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:27.895649 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7e8f6-7a10-45c5-8c82-8494c3afe1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:27.895765 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7e8f6-7a10-45c5-8c82-8494c3afe1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:27.960795 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b6d7e8f6-7a10-45c5-8c82-8494c3afe1d2 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:27.961984 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b6d7e8f6-7a10-45c5-8c82-8494c3afe1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:28.058555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 667/1319] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:28.356060 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb508f-2d3e-48d2-aba5-b814b77dd656 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:28.356373 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb508f-2d3e-48d2-aba5-b814b77dd656 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:28.356644 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb508f-2d3e-48d2-aba5-b814b77dd656 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:28.357984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 653/1320] 10.222.0.67 () {54 vars in 965 bytes} [Wed Aug 24 08:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:28.363828 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a95ef6-5a4c-487e-b8ac-28137ba12d05 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:28.364117 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a95ef6-5a4c-487e-b8ac-28137ba12d05 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:28.364411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a95ef6-5a4c-487e-b8ac-28137ba12d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:28.386792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:28.440424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-50a95ef6-5a4c-487e-b8ac-28137ba12d05 None None] MFA Rules not processed for user `2ee2e93901914b1d82607a3ae0e2acf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:28.441621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-50a95ef6-5a4c-487e-b8ac-28137ba12d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:28.516392 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 668/1321] 10.222.0.67 () {58 vars in 1062 bytes} [Wed Aug 24 08:47:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:28.525378 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb508f-2d3e-48d2-aba5-b814b77dd656 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:28.526900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-40bb508f-2d3e-48d2-aba5-b814b77dd656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:28.630208 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb508f-2d3e-48d2-aba5-b814b77dd656 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:28.703684 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff8588-e547-4ab3-8ce0-4936292fb446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:28.704317 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff8588-e547-4ab3-8ce0-4936292fb446 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:28.704600 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff8588-e547-4ab3-8ce0-4936292fb446 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:28.704877 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff8588-e547-4ab3-8ce0-4936292fb446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:28.706466 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f4ff8588-e547-4ab3-8ce0-4936292fb446 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:28.713886 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:28.816650 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff8588-e547-4ab3-8ce0-4936292fb446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:28.818089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff8588-e547-4ab3-8ce0-4936292fb446 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:28.836181 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 654/1322] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:31.905372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e18087-1704-460d-9f29-c949b32537f0 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:31.905933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e18087-1704-460d-9f29-c949b32537f0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:31.905933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e18087-1704-460d-9f29-c949b32537f0 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:31.908135 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 669/1323] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:47:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:31.913489 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc17f64-66d2-4f6c-8d4d-848ad36f428f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:31.913920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc17f64-66d2-4f6c-8d4d-848ad36f428f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:31.914071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc17f64-66d2-4f6c-8d4d-848ad36f428f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:31.980699 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7dc17f64-66d2-4f6c-8d4d-848ad36f428f None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:31.981895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7dc17f64-66d2-4f6c-8d4d-848ad36f428f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:32.043321 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:32.084627 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 655/1324] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:32.365750 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e18087-1704-460d-9f29-c949b32537f0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:32.367272 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d6e18087-1704-460d-9f29-c949b32537f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:32.476119 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e18087-1704-460d-9f29-c949b32537f0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:32.489106 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:32.554328 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b048a-ca69-4de8-968c-8c6ca771a7f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:32.554854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b048a-ca69-4de8-968c-8c6ca771a7f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:32.555168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b048a-ca69-4de8-968c-8c6ca771a7f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:32.555400 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b048a-ca69-4de8-968c-8c6ca771a7f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:32.557016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d46b048a-ca69-4de8-968c-8c6ca771a7f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:32.669408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b048a-ca69-4de8-968c-8c6ca771a7f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:32.670862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b048a-ca69-4de8-968c-8c6ca771a7f2 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:32.689571 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 670/1325] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:34.679722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b819c-3399-4177-9072-9a64ef975960 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:34.680402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b819c-3399-4177-9072-9a64ef975960 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:34.680402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b819c-3399-4177-9072-9a64ef975960 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:34.681720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 656/1326] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:34.687274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a6cfb-2900-4352-9b6d-9eaeaf4eca56 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:34.687745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a6cfb-2900-4352-9b6d-9eaeaf4eca56 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:34.687866 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a6cfb-2900-4352-9b6d-9eaeaf4eca56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:34.739484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:34.758711 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-fa1a6cfb-2900-4352-9b6d-9eaeaf4eca56 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:34.760027 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fa1a6cfb-2900-4352-9b6d-9eaeaf4eca56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:34.858562 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 671/1327] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:35.140994 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b819c-3399-4177-9072-9a64ef975960 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:35.142786 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8c8b819c-3399-4177-9072-9a64ef975960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:35.247348 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b819c-3399-4177-9072-9a64ef975960 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:35.324768 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f58e3f-c646-4c75-b256-dc6a663df071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:35.325377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f58e3f-c646-4c75-b256-dc6a663df071 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:35.325652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f58e3f-c646-4c75-b256-dc6a663df071 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:35.325917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f58e3f-c646-4c75-b256-dc6a663df071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:35.327486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f2f58e3f-c646-4c75-b256-dc6a663df071 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:35.345774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:35.441250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f58e3f-c646-4c75-b256-dc6a663df071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:35.442700 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f58e3f-c646-4c75-b256-dc6a663df071 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:35.463816 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 657/1328] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:37.432506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3201696-54e3-4911-88ec-1f26ff3d7a72 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:37.433011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3201696-54e3-4911-88ec-1f26ff3d7a72 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:37.433011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3201696-54e3-4911-88ec-1f26ff3d7a72 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:37.435203 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 672/1329] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:47:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:37.440395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7626e9a-7cec-4e24-9bd3-40655c9c5be7 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:37.440900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7626e9a-7cec-4e24-9bd3-40655c9c5be7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:37.440983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7626e9a-7cec-4e24-9bd3-40655c9c5be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:37.511814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b7626e9a-7cec-4e24-9bd3-40655c9c5be7 None None] MFA Rules not processed for user `98c373122e5c4b1a947b0dbd458f5b95`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:37.513062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b7626e9a-7cec-4e24-9bd3-40655c9c5be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:37.583016 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:37.616000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 658/1330] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:37 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:47:37.905269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3201696-54e3-4911-88ec-1f26ff3d7a72 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:37.906796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c3201696-54e3-4911-88ec-1f26ff3d7a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:38.037480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3201696-54e3-4911-88ec-1f26ff3d7a72 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:38.059746 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:38.117211 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c218e510-a1ee-4de7-ad89-78d0df205fc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:38.117832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218e510-a1ee-4de7-ad89-78d0df205fc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:38.118114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218e510-a1ee-4de7-ad89-78d0df205fc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:38.118382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218e510-a1ee-4de7-ad89-78d0df205fc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:38.119977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c218e510-a1ee-4de7-ad89-78d0df205fc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:38.237638 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c218e510-a1ee-4de7-ad89-78d0df205fc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:38.238979 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c218e510-a1ee-4de7-ad89-78d0df205fc1 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:38.257464 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 673/1331] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:37 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:40.401990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601a33d-0fc2-40b0-96fc-a35e695e2aa8 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:40.402530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601a33d-0fc2-40b0-96fc-a35e695e2aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:40.402530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601a33d-0fc2-40b0-96fc-a35e695e2aa8 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:40.403889 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 659/1332] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:40.410044 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6ab1d-757f-44f7-8741-3cba1fca536b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:40.410453 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6ab1d-757f-44f7-8741-3cba1fca536b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:40.410617 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6ab1d-757f-44f7-8741-3cba1fca536b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:40.475242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:40.478220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3ef6ab1d-757f-44f7-8741-3cba1fca536b None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:40.479975 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3ef6ab1d-757f-44f7-8741-3cba1fca536b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:40.576216 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 674/1333] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:40.879889 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601a33d-0fc2-40b0-96fc-a35e695e2aa8 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:40.881477 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9601a33d-0fc2-40b0-96fc-a35e695e2aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:40.990087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601a33d-0fc2-40b0-96fc-a35e695e2aa8 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:41.066775 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4729e7e-89fd-42a6-9900-0b3f7a52be85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:41.067308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4729e7e-89fd-42a6-9900-0b3f7a52be85 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:41.067586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4729e7e-89fd-42a6-9900-0b3f7a52be85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:41.067855 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4729e7e-89fd-42a6-9900-0b3f7a52be85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:41.069522 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c4729e7e-89fd-42a6-9900-0b3f7a52be85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:41.105488 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:41.205702 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4729e7e-89fd-42a6-9900-0b3f7a52be85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:41.207207 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4729e7e-89fd-42a6-9900-0b3f7a52be85 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:41.227274 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 660/1334] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:43.434499 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f011cb-eed8-4401-bfa1-3acc8afcd418 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:43.435078 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f011cb-eed8-4401-bfa1-3acc8afcd418 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:43.435078 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f011cb-eed8-4401-bfa1-3acc8afcd418 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:43.436404 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 675/1335] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:43.441998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1a75a-f5fd-4755-b5fb-29c3b74c4935 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:43.442481 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1a75a-f5fd-4755-b5fb-29c3b74c4935 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:43.442666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1a75a-f5fd-4755-b5fb-29c3b74c4935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:43.508656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d0b1a75a-f5fd-4755-b5fb-29c3b74c4935 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:43.510027 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d0b1a75a-f5fd-4755-b5fb-29c3b74c4935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:43.583874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:43.606807 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 661/1336] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:43.900178 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f011cb-eed8-4401-bfa1-3acc8afcd418 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:43.901698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c8f011cb-eed8-4401-bfa1-3acc8afcd418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:44.006932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f011cb-eed8-4401-bfa1-3acc8afcd418 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:44.035513 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:44.081146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574299e-751b-484e-bf98-b62891dabf4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:44.081760 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574299e-751b-484e-bf98-b62891dabf4a service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:44.082040 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574299e-751b-484e-bf98-b62891dabf4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:44.082308 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574299e-751b-484e-bf98-b62891dabf4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:44.083900 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6574299e-751b-484e-bf98-b62891dabf4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:44.196180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574299e-751b-484e-bf98-b62891dabf4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:44.197610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574299e-751b-484e-bf98-b62891dabf4a service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:44.216427 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 676/1337] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:46.855949 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba4bd4-0a60-4d08-9263-9cafc6beaec8 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:46.856560 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba4bd4-0a60-4d08-9263-9cafc6beaec8 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:46.856560 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba4bd4-0a60-4d08-9263-9cafc6beaec8 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:46.857936 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 662/1338] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:46.864187 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35ae38-db1d-4990-abb4-ba6fb3859640 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:46.864707 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35ae38-db1d-4990-abb4-ba6fb3859640 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:46.864819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35ae38-db1d-4990-abb4-ba6fb3859640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:46.955732 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-cc35ae38-db1d-4990-abb4-ba6fb3859640 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:46.956975 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cc35ae38-db1d-4990-abb4-ba6fb3859640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:46.966385 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:47.066233 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 677/1339] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:47:47.363066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba4bd4-0a60-4d08-9263-9cafc6beaec8 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:47.364642 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8bba4bd4-0a60-4d08-9263-9cafc6beaec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:47.495305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba4bd4-0a60-4d08-9263-9cafc6beaec8 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:47.569012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7740f3c-2ce4-49a5-8c7c-b220c6a17013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:47.569622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7740f3c-2ce4-49a5-8c7c-b220c6a17013 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:47.569897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7740f3c-2ce4-49a5-8c7c-b220c6a17013 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:47.570176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7740f3c-2ce4-49a5-8c7c-b220c6a17013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:47.572026 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a7740f3c-2ce4-49a5-8c7c-b220c6a17013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:47.607463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:47.692232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7740f3c-2ce4-49a5-8c7c-b220c6a17013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:47.694686 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7740f3c-2ce4-49a5-8c7c-b220c6a17013 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:47.716152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 663/1340] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:51.607603 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5edc8c-03a8-47b5-880e-f5c035dcac79 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:51.607982 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5edc8c-03a8-47b5-880e-f5c035dcac79 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:51.608282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5edc8c-03a8-47b5-880e-f5c035dcac79 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:51.609551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 678/1341] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:51.615274 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85deb6d-4d6f-435a-b05d-197a2480a8e5 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:51.615753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85deb6d-4d6f-435a-b05d-197a2480a8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:51.615858 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85deb6d-4d6f-435a-b05d-197a2480a8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:51.684500 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-a85deb6d-4d6f-435a-b05d-197a2480a8e5 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:51.685698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a85deb6d-4d6f-435a-b05d-197a2480a8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:51.782552 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:51.797137 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 664/1342] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:52.082209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5edc8c-03a8-47b5-880e-f5c035dcac79 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:52.083761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9e5edc8c-03a8-47b5-880e-f5c035dcac79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:52.196701 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5edc8c-03a8-47b5-880e-f5c035dcac79 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:52.237871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:52.273151 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724e84d-ec3a-4f6d-9bd0-d7892677befd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:52.273761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724e84d-ec3a-4f6d-9bd0-d7892677befd service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:52.274034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724e84d-ec3a-4f6d-9bd0-d7892677befd service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:52.274298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724e84d-ec3a-4f6d-9bd0-d7892677befd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:52.275883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4724e84d-ec3a-4f6d-9bd0-d7892677befd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:52.396779 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724e84d-ec3a-4f6d-9bd0-d7892677befd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:52.398226 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724e84d-ec3a-4f6d-9bd0-d7892677befd service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:52.416529 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 679/1343] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:54.918704 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9bc7a-ba7a-4a26-9763-59fc55d0d5c2 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:54.919271 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9bc7a-ba7a-4a26-9763-59fc55d0d5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:54.919271 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9bc7a-ba7a-4a26-9763-59fc55d0d5c2 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:54.920636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 665/1344] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:54.927035 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae1710-4cd0-4ad1-8407-b31a68877164 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:54.927581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae1710-4cd0-4ad1-8407-b31a68877164 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:54.927727 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae1710-4cd0-4ad1-8407-b31a68877164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:55.005440 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2cae1710-4cd0-4ad1-8407-b31a68877164 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:55.006582 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2cae1710-4cd0-4ad1-8407-b31a68877164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:55.026850 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:55.123293 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 680/1345] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:47:55.409181 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9bc7a-ba7a-4a26-9763-59fc55d0d5c2 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:47:55.411800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a4d9bc7a-ba7a-4a26-9763-59fc55d0d5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:55.554202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9bc7a-ba7a-4a26-9763-59fc55d0d5c2 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:47:55.646928 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f7eba-1e5a-4e62-afca-1796c2ce6249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:47:55.647577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f7eba-1e5a-4e62-afca-1796c2ce6249 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:55.647873 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f7eba-1e5a-4e62-afca-1796c2ce6249 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:55.648151 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f7eba-1e5a-4e62-afca-1796c2ce6249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:55.649832 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9c0f7eba-1e5a-4e62-afca-1796c2ce6249 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:47:55.700403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:47:55.788812 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f7eba-1e5a-4e62-afca-1796c2ce6249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:47:55.790264 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f7eba-1e5a-4e62-afca-1796c2ce6249 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:47:55.813514 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 666/1346] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:47:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:47:59.898069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffb335-92ac-41aa-8060-02c0d45a03fb None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:59.898805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffb335-92ac-41aa-8060-02c0d45a03fb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:59.898805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffb335-92ac-41aa-8060-02c0d45a03fb None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:59.899996 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 681/1347] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:47:59.906644 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa6be3-6e2a-42ea-845b-85ded149e533 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:47:59.909844 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa6be3-6e2a-42ea-845b-85ded149e533 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:47:59.910360 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa6be3-6e2a-42ea-845b-85ded149e533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:47:59.996067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9daa6be3-6e2a-42ea-845b-85ded149e533 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:47:59.997556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9daa6be3-6e2a-42ea-845b-85ded149e533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:00.120030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:00.126671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 667/1348] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:47:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:00.406832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffb335-92ac-41aa-8060-02c0d45a03fb None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:00.408411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-45ffb335-92ac-41aa-8060-02c0d45a03fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:00.520892 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffb335-92ac-41aa-8060-02c0d45a03fb None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:00.570129 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:00.597815 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19dbfbb-9453-48b1-abc5-6f8b96509b06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:00.598458 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dbfbb-9453-48b1-abc5-6f8b96509b06 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:00.598736 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dbfbb-9453-48b1-abc5-6f8b96509b06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:00.599002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dbfbb-9453-48b1-abc5-6f8b96509b06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:00.600775 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f19dbfbb-9453-48b1-abc5-6f8b96509b06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:00.711563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19dbfbb-9453-48b1-abc5-6f8b96509b06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:00.713028 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19dbfbb-9453-48b1-abc5-6f8b96509b06 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:00.731415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 682/1349] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:05.660462 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14d94b-2300-4c32-b32a-bf03d27d47e3 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:05.661908 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14d94b-2300-4c32-b32a-bf03d27d47e3 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:05.662403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14d94b-2300-4c32-b32a-bf03d27d47e3 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:05.664302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 668/1350] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 24 08:48:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:48:05.669698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6e2e1-4e6c-4172-b71e-17eaa7b83610 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:05.670612 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6e2e1-4e6c-4172-b71e-17eaa7b83610 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:05.671113 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6e2e1-4e6c-4172-b71e-17eaa7b83610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:05.771473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e1f6e2e1-4e6c-4172-b71e-17eaa7b83610 None None] MFA Rules not processed for user `33a0844a1c8f4ea4932e12d53e5ee9ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:05.773501 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e1f6e2e1-4e6c-4172-b71e-17eaa7b83610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:05.796237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:05.960405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 683/1351] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 24 08:48:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:10.047777 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4eacc-ce24-4d7e-934e-59c10b931342 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:10.048386 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4eacc-ce24-4d7e-934e-59c10b931342 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:10.048386 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4eacc-ce24-4d7e-934e-59c10b931342 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:10.050588 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 669/1352] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 24 08:48:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:10.055545 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57bd9f-c61f-4259-bd3a-cb21746e8d70 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:10.056009 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57bd9f-c61f-4259-bd3a-cb21746e8d70 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:10.056088 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57bd9f-c61f-4259-bd3a-cb21746e8d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:10.138457 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7b57bd9f-c61f-4259-bd3a-cb21746e8d70 None None] MFA Rules not processed for user `33a0844a1c8f4ea4932e12d53e5ee9ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:10.139691 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7b57bd9f-c61f-4259-bd3a-cb21746e8d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:10.219087 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:10.257844 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 684/1353] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 24 08:48:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:11.816779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317c80d-c1fe-45b3-a90b-abb72ed9e5f1 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:11.817335 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317c80d-c1fe-45b3-a90b-abb72ed9e5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:11.817335 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317c80d-c1fe-45b3-a90b-abb72ed9e5f1 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:11.818708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 670/1354] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 24 08:48:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:11.824338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203d6aa-a559-4b64-8746-4b65a475a359 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:11.824822 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203d6aa-a559-4b64-8746-4b65a475a359 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:11.824904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203d6aa-a559-4b64-8746-4b65a475a359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:11.906638 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c203d6aa-a559-4b64-8746-4b65a475a359 None None] MFA Rules not processed for user `33a0844a1c8f4ea4932e12d53e5ee9ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:11.908005 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c203d6aa-a559-4b64-8746-4b65a475a359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:12.016659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 685/1355] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 24 08:48:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:15.453291 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad53842-fb2d-4c26-9483-154b549bddda None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:15.455503 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad53842-fb2d-4c26-9483-154b549bddda None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:15.455503 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad53842-fb2d-4c26-9483-154b549bddda None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:15.455767 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 671/1356] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:15.461180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb18b2-472b-4b3a-9f90-a13a9c1d34fc None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:15.461634 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb18b2-472b-4b3a-9f90-a13a9c1d34fc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:15.461838 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb18b2-472b-4b3a-9f90-a13a9c1d34fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:15.514364 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:15.550785 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1fcb18b2-472b-4b3a-9f90-a13a9c1d34fc None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:15.552080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1fcb18b2-472b-4b3a-9f90-a13a9c1d34fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:15.701590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 686/1357] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:48:16.012396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea6fbb-ff35-4a60-bd57-44932ca3b4bf None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:16.012966 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea6fbb-ff35-4a60-bd57-44932ca3b4bf None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:16.013450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea6fbb-ff35-4a60-bd57-44932ca3b4bf None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:16.015270 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 672/1358] 10.222.0.67 () {54 vars in 962 bytes} [Wed Aug 24 08:48:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:16.032128 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168afcb5-7589-410a-a578-edaa80e352c8 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:16.032408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168afcb5-7589-410a-a578-edaa80e352c8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:16.032681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168afcb5-7589-410a-a578-edaa80e352c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:16.115805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-168afcb5-7589-410a-a578-edaa80e352c8 None None] MFA Rules not processed for user `ca56dd7216fb4946946b2801e144fec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:16.117158 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-168afcb5-7589-410a-a578-edaa80e352c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:16.179036 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:16.251718 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 687/1359] 10.222.0.67 () {58 vars in 1059 bytes} [Wed Aug 24 08:48:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:16.259697 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea6fbb-ff35-4a60-bd57-44932ca3b4bf None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:16.261506 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f9ea6fbb-ff35-4a60-bd57-44932ca3b4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:16.427903 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea6fbb-ff35-4a60-bd57-44932ca3b4bf None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:16.510273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:16.555095 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9fa84e-49bc-43ba-aa46-cd606aa584c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:16.555877 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fa84e-49bc-43ba-aa46-cd606aa584c8 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:16.556289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fa84e-49bc-43ba-aa46-cd606aa584c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:16.556664 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fa84e-49bc-43ba-aa46-cd606aa584c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:16.558511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fb9fa84e-49bc-43ba-aa46-cd606aa584c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:16.658667 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9fa84e-49bc-43ba-aa46-cd606aa584c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:16.660413 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9fa84e-49bc-43ba-aa46-cd606aa584c8 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:16.679614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 673/1360] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:18.842979 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354870a-f695-4faa-be30-e047884c2221 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:18.843508 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354870a-f695-4faa-be30-e047884c2221 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:18.843508 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354870a-f695-4faa-be30-e047884c2221 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:18.844984 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 688/1361] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:18.850364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535af0bb-bffb-4082-91ce-d52386320168 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:18.850840 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535af0bb-bffb-4082-91ce-d52386320168 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:18.850949 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535af0bb-bffb-4082-91ce-d52386320168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:18.921747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-535af0bb-bffb-4082-91ce-d52386320168 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:18.922958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-535af0bb-bffb-4082-91ce-d52386320168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:18.963384 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:19.036377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 674/1362] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:19.327649 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354870a-f695-4faa-be30-e047884c2221 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:19.329281 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6354870a-f695-4faa-be30-e047884c2221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:19.486913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354870a-f695-4faa-be30-e047884c2221 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:19.496979 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:19.614792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d38dd7-bb36-439a-a95c-61f49be09807 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:19.615411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d38dd7-bb36-439a-a95c-61f49be09807 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:19.615693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d38dd7-bb36-439a-a95c-61f49be09807 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:19.616065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d38dd7-bb36-439a-a95c-61f49be09807 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:19.617642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-94d38dd7-bb36-439a-a95c-61f49be09807 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:19.732677 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d38dd7-bb36-439a-a95c-61f49be09807 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:19.734273 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d38dd7-bb36-439a-a95c-61f49be09807 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:19.759310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 689/1363] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:22.045351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582942a9-2218-45a5-a0c0-d1f38b2a32b0 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:22.046209 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582942a9-2218-45a5-a0c0-d1f38b2a32b0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:22.046494 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582942a9-2218-45a5-a0c0-d1f38b2a32b0 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:22.047868 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 675/1364] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:48:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:22.053581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb5e01-e308-4ac6-b2dd-7080c4252027 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:22.054052 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb5e01-e308-4ac6-b2dd-7080c4252027 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:22.054132 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb5e01-e308-4ac6-b2dd-7080c4252027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:22.070919 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:22.125477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-99fb5e01-e308-4ac6-b2dd-7080c4252027 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:22.126718 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-99fb5e01-e308-4ac6-b2dd-7080c4252027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:22.245588 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 690/1365] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:22.544030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f90762-577d-44df-810c-0a0e766bd0a1 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:22.544376 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f90762-577d-44df-810c-0a0e766bd0a1 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:22.544760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f90762-577d-44df-810c-0a0e766bd0a1 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:22.546098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 676/1366] 10.222.0.67 () {54 vars in 962 bytes} [Wed Aug 24 08:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:22.552352 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d31db4-691f-44e0-bcf9-32f812b0ea17 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:22.552593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d31db4-691f-44e0-bcf9-32f812b0ea17 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:22.552845 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d31db4-691f-44e0-bcf9-32f812b0ea17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:22.644716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d2d31db4-691f-44e0-bcf9-32f812b0ea17 None None] MFA Rules not processed for user `ca56dd7216fb4946946b2801e144fec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:22.645970 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d2d31db4-691f-44e0-bcf9-32f812b0ea17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:22.684221 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:22.788404 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 691/1367] 10.222.0.67 () {58 vars in 1059 bytes} [Wed Aug 24 08:48:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:22.795048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f90762-577d-44df-810c-0a0e766bd0a1 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:22.796677 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-85f90762-577d-44df-810c-0a0e766bd0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:22.959720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f90762-577d-44df-810c-0a0e766bd0a1 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:22.982052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:23.089129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98ab7c-4aff-47b5-838f-2d8055d90559 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:23.089811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98ab7c-4aff-47b5-838f-2d8055d90559 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:23.090082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98ab7c-4aff-47b5-838f-2d8055d90559 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:23.090272 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98ab7c-4aff-47b5-838f-2d8055d90559 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:23.091863 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fa98ab7c-4aff-47b5-838f-2d8055d90559 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:23.187622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98ab7c-4aff-47b5-838f-2d8055d90559 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:23.189303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98ab7c-4aff-47b5-838f-2d8055d90559 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:23.211255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 677/1368] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:23.785136 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a19e77-80a5-433a-a856-3dc66ee8ef82 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:23.785707 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a19e77-80a5-433a-a856-3dc66ee8ef82 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:23.785707 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a19e77-80a5-433a-a856-3dc66ee8ef82 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:23.787205 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 692/1369] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:23.792537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e8901-a8bb-4a31-aab5-808c7b32853a None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:23.792827 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e8901-a8bb-4a31-aab5-808c7b32853a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:23.793097 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e8901-a8bb-4a31-aab5-808c7b32853a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:23.829232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:23.869494 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d25e8901-a8bb-4a31-aab5-808c7b32853a None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:23.870742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d25e8901-a8bb-4a31-aab5-808c7b32853a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:23.974423 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 678/1370] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:24.561757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36790f5c-fcd2-42ca-a3b7-ef995c5d6102 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:24.562135 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36790f5c-fcd2-42ca-a3b7-ef995c5d6102 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:24.562481 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36790f5c-fcd2-42ca-a3b7-ef995c5d6102 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:24.564069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 693/1371] 10.222.0.67 () {54 vars in 964 bytes} [Wed Aug 24 08:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:24.569405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288f6d1-12eb-48fb-a696-4b25b712735f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:24.569835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288f6d1-12eb-48fb-a696-4b25b712735f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:24.570027 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288f6d1-12eb-48fb-a696-4b25b712735f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:24.656505 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9288f6d1-12eb-48fb-a696-4b25b712735f None None] MFA Rules not processed for user `5de1586efc8c4dec898b0fb54b56acf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:24.658030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9288f6d1-12eb-48fb-a696-4b25b712735f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:24.708130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:24.740491 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 679/1372] 10.222.0.67 () {58 vars in 1061 bytes} [Wed Aug 24 08:48:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:24.751359 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36790f5c-fcd2-42ca-a3b7-ef995c5d6102 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:24.752981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-36790f5c-fcd2-42ca-a3b7-ef995c5d6102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:24.864497 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36790f5c-fcd2-42ca-a3b7-ef995c5d6102 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:24.907626 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:24.948114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65132f-abc5-4e38-9345-0abfc05885e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:24.949667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65132f-abc5-4e38-9345-0abfc05885e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:24.950215 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65132f-abc5-4e38-9345-0abfc05885e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:24.950691 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65132f-abc5-4e38-9345-0abfc05885e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:24.952985 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4b65132f-abc5-4e38-9345-0abfc05885e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:25.080438 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65132f-abc5-4e38-9345-0abfc05885e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:25.081749 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65132f-abc5-4e38-9345-0abfc05885e6 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:25.114181 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 694/1373] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:25.248841 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8355fda-5279-4c23-ba14-ad21ab21e281 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:25.249129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8355fda-5279-4c23-ba14-ad21ab21e281 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:25.249429 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8355fda-5279-4c23-ba14-ad21ab21e281 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:25.250969 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 680/1374] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:25.256551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634012eb-a9ac-493e-9984-7437cfafeaa7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:25.256841 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634012eb-a9ac-493e-9984-7437cfafeaa7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:25.257297 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634012eb-a9ac-493e-9984-7437cfafeaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:25.328308 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-634012eb-a9ac-493e-9984-7437cfafeaa7 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:25.329522 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-634012eb-a9ac-493e-9984-7437cfafeaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:25.350788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:25.430927 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 695/1375] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:25.730209 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8355fda-5279-4c23-ba14-ad21ab21e281 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:25.731795 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f8355fda-5279-4c23-ba14-ad21ab21e281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:25.894957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8355fda-5279-4c23-ba14-ad21ab21e281 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:25.949639 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:26.014480 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e848b-a1f3-4751-ab35-e10d9791ce85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:26.014911 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e848b-a1f3-4751-ab35-e10d9791ce85 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:26.015189 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e848b-a1f3-4751-ab35-e10d9791ce85 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:26.015455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e848b-a1f3-4751-ab35-e10d9791ce85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:26.017094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-514e848b-a1f3-4751-ab35-e10d9791ce85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:26.106621 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e848b-a1f3-4751-ab35-e10d9791ce85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:26.108296 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e848b-a1f3-4751-ab35-e10d9791ce85 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:26.126578 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 681/1376] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:28.103511 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eaba2f-26f2-4207-870d-d38a6773dde5 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:28.104015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eaba2f-26f2-4207-870d-d38a6773dde5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:28.104173 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eaba2f-26f2-4207-870d-d38a6773dde5 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:28.108193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 696/1377] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:28.113714 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7c2d4-3d78-412c-887f-f29aa38269e1 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:28.114327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7c2d4-3d78-412c-887f-f29aa38269e1 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:28.114327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7c2d4-3d78-412c-887f-f29aa38269e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:28.180510 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-a3c7c2d4-3d78-412c-887f-f29aa38269e1 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:28.181726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a3c7c2d4-3d78-412c-887f-f29aa38269e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:28.188441 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:28.306874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 682/1378] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:28.600257 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eaba2f-26f2-4207-870d-d38a6773dde5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:28.601762 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c8eaba2f-26f2-4207-870d-d38a6773dde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:28.798450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eaba2f-26f2-4207-870d-d38a6773dde5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:28.804911 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:28.945539 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebcb15-0657-43cb-a59a-aea26209bb1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:28.946154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebcb15-0657-43cb-a59a-aea26209bb1e service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:28.946438 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebcb15-0657-43cb-a59a-aea26209bb1e service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:28.946705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebcb15-0657-43cb-a59a-aea26209bb1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:28.948312 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9eebcb15-0657-43cb-a59a-aea26209bb1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:29.048274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebcb15-0657-43cb-a59a-aea26209bb1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:29.049885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebcb15-0657-43cb-a59a-aea26209bb1e service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:29.078949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 697/1379] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:31.067652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08045c74-01ae-4293-8ded-ababf9758d9f None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:31.068191 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08045c74-01ae-4293-8ded-ababf9758d9f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:31.068191 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08045c74-01ae-4293-8ded-ababf9758d9f None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:31.069657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 683/1380] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:31.075237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030449db-bd55-46dd-94e3-3dd6f2ed3f36 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:31.075679 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030449db-bd55-46dd-94e3-3dd6f2ed3f36 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:31.075784 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030449db-bd55-46dd-94e3-3dd6f2ed3f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:31.087764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:31.150819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-030449db-bd55-46dd-94e3-3dd6f2ed3f36 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:31.152013 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-030449db-bd55-46dd-94e3-3dd6f2ed3f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:31.264405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 698/1381] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:31.570201 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08045c74-01ae-4293-8ded-ababf9758d9f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:31.571701 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-08045c74-01ae-4293-8ded-ababf9758d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:31.757439 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:31.778032 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08045c74-01ae-4293-8ded-ababf9758d9f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:31.929196 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eadcfaf-45d1-4366-96b6-43c3984300e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:31.930018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadcfaf-45d1-4366-96b6-43c3984300e4 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:31.930403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadcfaf-45d1-4366-96b6-43c3984300e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:31.930753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadcfaf-45d1-4366-96b6-43c3984300e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:31.932514 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9eadcfaf-45d1-4366-96b6-43c3984300e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:32.041529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eadcfaf-45d1-4366-96b6-43c3984300e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:32.043208 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eadcfaf-45d1-4366-96b6-43c3984300e4 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:32.055614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:32.065805 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 684/1382] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:48:34.093385 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093392f-8a5d-4dfd-b8bc-f1a9656c86d7 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:34.093385 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093392f-8a5d-4dfd-b8bc-f1a9656c86d7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:34.093385 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093392f-8a5d-4dfd-b8bc-f1a9656c86d7 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:34.094781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 699/1383] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:34.101022 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe0d7b-6db8-4b01-a962-34211427bb56 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:34.101562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe0d7b-6db8-4b01-a962-34211427bb56 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:34.101562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe0d7b-6db8-4b01-a962-34211427bb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:34.194158 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-8abe0d7b-6db8-4b01-a962-34211427bb56 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:34.196254 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8abe0d7b-6db8-4b01-a962-34211427bb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:34.323166 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 685/1384] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:34.619568 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093392f-8a5d-4dfd-b8bc-f1a9656c86d7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:34.621152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e093392f-8a5d-4dfd-b8bc-f1a9656c86d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:34.711502 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:34.777700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093392f-8a5d-4dfd-b8bc-f1a9656c86d7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:34.910950 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f46724-5b0a-4ec2-a47c-3988bb7bc262 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:34.911636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f46724-5b0a-4ec2-a47c-3988bb7bc262 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:34.911838 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f46724-5b0a-4ec2-a47c-3988bb7bc262 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:34.912140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f46724-5b0a-4ec2-a47c-3988bb7bc262 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:34.913750 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-97f46724-5b0a-4ec2-a47c-3988bb7bc262 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:34.973161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:35.023375 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f46724-5b0a-4ec2-a47c-3988bb7bc262 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:35.025396 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f46724-5b0a-4ec2-a47c-3988bb7bc262 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:35.048546 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 700/1385] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:37.348329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c63ff3-f638-43fe-adb2-db574001c5af None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:37.349052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c63ff3-f638-43fe-adb2-db574001c5af None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:37.349052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c63ff3-f638-43fe-adb2-db574001c5af None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:37.350416 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 686/1386] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:37.356782 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60992ba-962b-421a-9867-5c606ac0a827 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:37.357399 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60992ba-962b-421a-9867-5c606ac0a827 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:37.357399 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60992ba-962b-421a-9867-5c606ac0a827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:37.434871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b60992ba-962b-421a-9867-5c606ac0a827 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:37.436161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b60992ba-962b-421a-9867-5c606ac0a827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:37.552790 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 701/1387] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:48:37.953572 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c63ff3-f638-43fe-adb2-db574001c5af None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:37.956125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-69c63ff3-f638-43fe-adb2-db574001c5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:38.075438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:38.167400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c63ff3-f638-43fe-adb2-db574001c5af None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:38.308517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fcdeb-66de-4114-aa90-bbb7539eab5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:38.309137 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fcdeb-66de-4114-aa90-bbb7539eab5a service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:38.309415 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fcdeb-66de-4114-aa90-bbb7539eab5a service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:38.309682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fcdeb-66de-4114-aa90-bbb7539eab5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:38.311329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-211fcdeb-66de-4114-aa90-bbb7539eab5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:38.355406 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:38.422331 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211fcdeb-66de-4114-aa90-bbb7539eab5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:38.424036 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211fcdeb-66de-4114-aa90-bbb7539eab5a service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:38.445397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 687/1388] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:40.433949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3cd8f-6896-4c16-9620-98ded4ee1019 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:40.434527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3cd8f-6896-4c16-9620-98ded4ee1019 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:40.434527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3cd8f-6896-4c16-9620-98ded4ee1019 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:40.435968 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 702/1389] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:40.441990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3a275-bf3c-4c41-9a08-99c40bb9967f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:40.442479 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3a275-bf3c-4c41-9a08-99c40bb9967f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:40.443031 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3a275-bf3c-4c41-9a08-99c40bb9967f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:40.514442 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-72b3a275-bf3c-4c41-9a08-99c40bb9967f None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:40.515661 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-72b3a275-bf3c-4c41-9a08-99c40bb9967f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:40.618991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 688/1390] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:40.929019 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3cd8f-6896-4c16-9620-98ded4ee1019 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:40.930881 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6ce3cd8f-6896-4c16-9620-98ded4ee1019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:40.941183 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6d2cf-d251-4d1c-be7c-0fd3c7e056e6 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:40.941500 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6d2cf-d251-4d1c-be7c-0fd3c7e056e6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:40.941768 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6d2cf-d251-4d1c-be7c-0fd3c7e056e6 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:40.943226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 689/1391] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:48:40.948323 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e3f37-ee83-4fe7-be68-11d9a02203b4 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:40.948599 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e3f37-ee83-4fe7-be68-11d9a02203b4 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:40.948865 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e3f37-ee83-4fe7-be68-11d9a02203b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:40.963866 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:40.964923 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:41.018128 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-950e3f37-ee83-4fe7-be68-11d9a02203b4 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:41.019252 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-950e3f37-ee83-4fe7-be68-11d9a02203b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:41.104062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3cd8f-6896-4c16-9620-98ded4ee1019 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:41.122552 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 690/1392] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:48:41.213126 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:41.225886 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6d7e3-35c0-4e7e-89f4-dae141e551ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:41.226493 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6d7e3-35c0-4e7e-89f4-dae141e551ec service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:41.226768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6d7e3-35c0-4e7e-89f4-dae141e551ec service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:41.227033 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6d7e3-35c0-4e7e-89f4-dae141e551ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:41.228631 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-01d6d7e3-35c0-4e7e-89f4-dae141e551ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:41.333752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6d7e3-35c0-4e7e-89f4-dae141e551ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:41.337677 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6d7e3-35c0-4e7e-89f4-dae141e551ec service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:41.366308 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 703/1393] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:41.440700 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7f752-128b-4894-8f15-b71c13a99478 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:41.441009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7f752-128b-4894-8f15-b71c13a99478 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:41.441483 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7f752-128b-4894-8f15-b71c13a99478 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:41.443509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 691/1394] 10.222.0.67 () {54 vars in 967 bytes} [Wed Aug 24 08:48:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:41.453472 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae3bf7-227e-4ac3-9ae9-160ca9767a1d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:41.454406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae3bf7-227e-4ac3-9ae9-160ca9767a1d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:41.454406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae3bf7-227e-4ac3-9ae9-160ca9767a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:41.565344 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-92ae3bf7-227e-4ac3-9ae9-160ca9767a1d None None] MFA Rules not processed for user `5979387f7aa54619b8d429368c306cc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:41.567339 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-92ae3bf7-227e-4ac3-9ae9-160ca9767a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:41.634698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:41.665163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 704/1395] 10.222.0.67 () {58 vars in 1064 bytes} [Wed Aug 24 08:48:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:41.673697 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7f752-128b-4894-8f15-b71c13a99478 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:41.675206 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-acf7f752-128b-4894-8f15-b71c13a99478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:41.786387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:41.794415 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7f752-128b-4894-8f15-b71c13a99478 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:41.885523 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c5a12-a2c2-4133-a42d-986af677a392 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5979387f7aa54619b8d429368c306cc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:41.886058 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c5a12-a2c2-4133-a42d-986af677a392 service glance] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:41.886338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c5a12-a2c2-4133-a42d-986af677a392 service glance] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:41.886610 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c5a12-a2c2-4133-a42d-986af677a392 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:41.888291 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fb8c5a12-a2c2-4133-a42d-986af677a392 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:42.025682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c5a12-a2c2-4133-a42d-986af677a392 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:42.027141 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c5a12-a2c2-4133-a42d-986af677a392 service glance] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:42.048082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 692/1396] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:42.835880 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0cb1d-48e7-44f2-ac2b-ef5065f46dac None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:42.836418 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0cb1d-48e7-44f2-ac2b-ef5065f46dac None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:42.836708 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0cb1d-48e7-44f2-ac2b-ef5065f46dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:42.905509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8cc0cb1d-48e7-44f2-ac2b-ef5065f46dac None None] MFA Rules not processed for user `b11abed8d4424a1e9957c3dc9aefadd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:42.906965 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8cc0cb1d-48e7-44f2-ac2b-ef5065f46dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:42.995349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 705/1397] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 24 08:48:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:43.042322 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c9ae4-af95-40cd-9937-b5812f9d23b7 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:43.042812 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c9ae4-af95-40cd-9937-b5812f9d23b7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:43.043016 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c9ae4-af95-40cd-9937-b5812f9d23b7 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:43.044471 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 693/1398] 10.222.0.67 () {54 vars in 943 bytes} [Wed Aug 24 08:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:43.051524 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f3d8a-7091-4192-837f-d40699a14f8e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:43.051889 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f3d8a-7091-4192-837f-d40699a14f8e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:43.052226 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f3d8a-7091-4192-837f-d40699a14f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:43.135188 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5f3f3d8a-7091-4192-837f-d40699a14f8e None None] MFA Rules not processed for user `46b1417d4ac24c8982b2052f4e99042a`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:43.136607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5f3f3d8a-7091-4192-837f-d40699a14f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:43.150719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:43.260599 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 706/1399] 10.222.0.67 () {58 vars in 1040 bytes} [Wed Aug 24 08:48:43 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:43.270518 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c9ae4-af95-40cd-9937-b5812f9d23b7 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:43.272119 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-722c9ae4-af95-40cd-9937-b5812f9d23b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:43.298850 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:43.421440 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c9ae4-af95-40cd-9937-b5812f9d23b7 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:43.436478 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603eb7bd-466f-4965-a413-9383e5ffe2ae None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:43.436868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603eb7bd-466f-4965-a413-9383e5ffe2ae None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:43.437234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603eb7bd-466f-4965-a413-9383e5ffe2ae None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:43.438660 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 707/1400] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:48:43.444168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4074170-e16f-4820-911d-85e3b971e3f5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:43.444567 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4074170-e16f-4820-911d-85e3b971e3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:43.444941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4074170-e16f-4820-911d-85e3b971e3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:43.516345 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f4074170-e16f-4820-911d-85e3b971e3f5 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:43.517583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f4074170-e16f-4820-911d-85e3b971e3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:43.530384 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7fd21-b0d6-440e-bbcf-f66a8e1b105b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46b1417d4ac24c8982b2052f4e99042a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:43.531039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7fd21-b0d6-440e-bbcf-f66a8e1b105b service swift] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:43.531260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7fd21-b0d6-440e-bbcf-f66a8e1b105b service swift] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:43.531528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7fd21-b0d6-440e-bbcf-f66a8e1b105b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:43.533184 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9dd7fd21-b0d6-440e-bbcf-f66a8e1b105b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:43.541222 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:43.600813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:43.621376 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7fd21-b0d6-440e-bbcf-f66a8e1b105b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:43.622976 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7fd21-b0d6-440e-bbcf-f66a8e1b105b service swift] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:43.624872 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 708/1401] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:48:43.632757 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 694/1402] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 24 08:48:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:48:43.919490 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4074170-e16f-4820-911d-85e3b971e3f5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:43.920745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f4074170-e16f-4820-911d-85e3b971e3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:44.049618 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4074170-e16f-4820-911d-85e3b971e3f5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:44.149899 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:44.178178 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81ebeb-dbac-4d23-b491-a86d7d58a3ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:44.178909 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81ebeb-dbac-4d23-b491-a86d7d58a3ae service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:44.179251 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81ebeb-dbac-4d23-b491-a86d7d58a3ae service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:44.179586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81ebeb-dbac-4d23-b491-a86d7d58a3ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:44.181340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ef81ebeb-dbac-4d23-b491-a86d7d58a3ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:44.283302 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81ebeb-dbac-4d23-b491-a86d7d58a3ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:44.285100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81ebeb-dbac-4d23-b491-a86d7d58a3ae service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:44.305992 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 709/1403] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:46.431905 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5da15-8511-487a-bbb8-1f43f5cc58a5 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:46.432570 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5da15-8511-487a-bbb8-1f43f5cc58a5 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:46.432570 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5da15-8511-487a-bbb8-1f43f5cc58a5 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:46.433834 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 695/1404] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:46.439224 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77eb36-af78-4b5f-9b38-0c220cf63744 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:46.439706 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77eb36-af78-4b5f-9b38-0c220cf63744 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:46.439806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77eb36-af78-4b5f-9b38-0c220cf63744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:46.518272 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4d77eb36-af78-4b5f-9b38-0c220cf63744 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:46.519496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4d77eb36-af78-4b5f-9b38-0c220cf63744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:46.578084 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:46.627696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 710/1405] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:46.921848 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5da15-8511-487a-bbb8-1f43f5cc58a5 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:46.923365 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e8f5da15-8511-487a-bbb8-1f43f5cc58a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:47.098886 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5da15-8511-487a-bbb8-1f43f5cc58a5 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:47.101500 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:47.252087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c024254-7ece-4483-bc76-1474a7814d83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:47.252791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c024254-7ece-4483-bc76-1474a7814d83 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:47.257490 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c024254-7ece-4483-bc76-1474a7814d83 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:47.257810 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c024254-7ece-4483-bc76-1474a7814d83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:47.259638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1c024254-7ece-4483-bc76-1474a7814d83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:47.373950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c024254-7ece-4483-bc76-1474a7814d83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:47.375902 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c024254-7ece-4483-bc76-1474a7814d83 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:47.402572 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 696/1406] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:49.389632 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c595a4-9bb7-411c-9b48-26808f38c0c1 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:49.390216 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c595a4-9bb7-411c-9b48-26808f38c0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:49.390216 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c595a4-9bb7-411c-9b48-26808f38c0c1 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:49.391542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 711/1407] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:48:49.398492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e0fe1-3c62-460c-9209-bf7cdcbbefe7 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:49.399063 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e0fe1-3c62-460c-9209-bf7cdcbbefe7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:49.399063 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e0fe1-3c62-460c-9209-bf7cdcbbefe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:49.409021 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:49.477864 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7e6e0fe1-3c62-460c-9209-bf7cdcbbefe7 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:48:49.479088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7e6e0fe1-3c62-460c-9209-bf7cdcbbefe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:49.590076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 697/1408] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:48:49.892664 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c595a4-9bb7-411c-9b48-26808f38c0c1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:48:49.894168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-54c595a4-9bb7-411c-9b48-26808f38c0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:50.078861 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c595a4-9bb7-411c-9b48-26808f38c0c1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:48:50.130301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:48:50.226008 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36044c25-4061-48d9-b2ee-29ae85e2e763 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:48:50.226741 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36044c25-4061-48d9-b2ee-29ae85e2e763 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:48:50.227020 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36044c25-4061-48d9-b2ee-29ae85e2e763 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:48:50.227284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36044c25-4061-48d9-b2ee-29ae85e2e763 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:48:50.228974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-36044c25-4061-48d9-b2ee-29ae85e2e763 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:48:50.337959 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36044c25-4061-48d9-b2ee-29ae85e2e763 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:48:50.339930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36044c25-4061-48d9-b2ee-29ae85e2e763 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:48:50.364859 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 712/1409] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:48:59.588258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=62 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:00.362077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=82 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:06.352275 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfcfc4-2592-4c4c-8a0f-cc4edb23560d None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:06.352851 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfcfc4-2592-4c4c-8a0f-cc4edb23560d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:06.352851 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfcfc4-2592-4c4c-8a0f-cc4edb23560d None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:06.354193 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 698/1410] 10.222.0.67 () {54 vars in 918 bytes} [Wed Aug 24 08:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:06.359458 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06b497-9643-4dc1-a244-07e44d9c3489 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:06.359892 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06b497-9643-4dc1-a244-07e44d9c3489 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:06.360004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06b497-9643-4dc1-a244-07e44d9c3489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:06.438219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8c06b497-9643-4dc1-a244-07e44d9c3489 None None] MFA Rules not processed for user `33a0844a1c8f4ea4932e12d53e5ee9ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:06.439411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8c06b497-9643-4dc1-a244-07e44d9c3489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:06.547790 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 713/1411] 10.222.0.67 () {58 vars in 1015 bytes} [Wed Aug 24 08:49:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:06.562904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51d096-fa27-4894-918d-6e21c3c448a8 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:06.563204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51d096-fa27-4894-918d-6e21c3c448a8 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:06.563470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51d096-fa27-4894-918d-6e21c3c448a8 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:06.564877 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 699/1412] 10.222.0.67 () {54 vars in 918 bytes} [Wed Aug 24 08:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:06.570439 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99ea5a-83d1-456f-ae38-7528b9a9a51a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:06.570723 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99ea5a-83d1-456f-ae38-7528b9a9a51a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:06.570995 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99ea5a-83d1-456f-ae38-7528b9a9a51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:06.643039 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:06.645963 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ba99ea5a-83d1-456f-ae38-7528b9a9a51a None None] MFA Rules not processed for user `ca56dd7216fb4946946b2801e144fec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:06.648040 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ba99ea5a-83d1-456f-ae38-7528b9a9a51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:06.764597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 714/1413] 10.222.0.67 () {58 vars in 1015 bytes} [Wed Aug 24 08:49:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:09.197766 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693b8f2-47aa-45de-bb18-a9e2a9da5de0 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:09.198339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693b8f2-47aa-45de-bb18-a9e2a9da5de0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:09.198339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693b8f2-47aa-45de-bb18-a9e2a9da5de0 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:09.199669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:09.205267 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf26a20-9fa7-44b7-80ed-125427874f2e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:09.205713 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf26a20-9fa7-44b7-80ed-125427874f2e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:09.205893 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf26a20-9fa7-44b7-80ed-125427874f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:09.273042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-daf26a20-9fa7-44b7-80ed-125427874f2e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:09.274419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-daf26a20-9fa7-44b7-80ed-125427874f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:09.336909 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:09.379528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:09.675245 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693b8f2-47aa-45de-bb18-a9e2a9da5de0 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:09.676795 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f693b8f2-47aa-45de-bb18-a9e2a9da5de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:09.823704 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693b8f2-47aa-45de-bb18-a9e2a9da5de0 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:09.900463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:09.940672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff0984-c867-4d3a-9669-cfa33da7dd7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:09.941205 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff0984-c867-4d3a-9669-cfa33da7dd7f service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:09.941486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff0984-c867-4d3a-9669-cfa33da7dd7f service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:09.941761 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff0984-c867-4d3a-9669-cfa33da7dd7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:09.943914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c1ff0984-c867-4d3a-9669-cfa33da7dd7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:10.035842 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff0984-c867-4d3a-9669-cfa33da7dd7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:10.037669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff0984-c867-4d3a-9669-cfa33da7dd7f service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:10.055678 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 701/1416] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:10.950689 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8e76c-96bc-4aa6-9cd3-e90d10307cb9 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:10.951251 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8e76c-96bc-4aa6-9cd3-e90d10307cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:10.951251 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8e76c-96bc-4aa6-9cd3-e90d10307cb9 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:10.952959 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:10.957862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9cb82-014f-40da-b46e-afe22065efea None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:10.958302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9cb82-014f-40da-b46e-afe22065efea None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:10.958400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9cb82-014f-40da-b46e-afe22065efea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:11.034383 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e4d9cb82-014f-40da-b46e-afe22065efea None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:11.035502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e4d9cb82-014f-40da-b46e-afe22065efea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:11.063059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:11.133665 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:11.283835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8e76c-96bc-4aa6-9cd3-e90d10307cb9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:11.285515 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a2c8e76c-96bc-4aa6-9cd3-e90d10307cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:11.445384 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8e76c-96bc-4aa6-9cd3-e90d10307cb9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:11.486471 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:11.560561 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a48e3a-d18a-48ed-9a13-1b6e119dda2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:11.561086 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a48e3a-d18a-48ed-9a13-1b6e119dda2e service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:11.561366 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a48e3a-d18a-48ed-9a13-1b6e119dda2e service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:11.561632 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a48e3a-d18a-48ed-9a13-1b6e119dda2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:11.563223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-67a48e3a-d18a-48ed-9a13-1b6e119dda2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:11.654316 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a48e3a-d18a-48ed-9a13-1b6e119dda2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:11.656118 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a48e3a-d18a-48ed-9a13-1b6e119dda2e service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:11.675993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 717/1419] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:12.584277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f4591-1d36-4183-ab60-174572301f4b None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:12.584826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f4591-1d36-4183-ab60-174572301f4b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:12.584826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f4591-1d36-4183-ab60-174572301f4b None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:12.586196 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:12.591180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bc99a-d178-4a62-95d6-6e120eeab49b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:12.591618 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bc99a-d178-4a62-95d6-6e120eeab49b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:12.591725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bc99a-d178-4a62-95d6-6e120eeab49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:12.648661 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:12.670455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b80bc99a-d178-4a62-95d6-6e120eeab49b None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:12.671658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b80bc99a-d178-4a62-95d6-6e120eeab49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:12.768422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:12.919671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f4591-1d36-4183-ab60-174572301f4b None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:12.921200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-412f4591-1d36-4183-ab60-174572301f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:13.070708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f4591-1d36-4183-ab60-174572301f4b None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:13.083115 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:13.187102 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399aac01-b223-4318-85fc-9b831ff3f861 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:13.187610 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399aac01-b223-4318-85fc-9b831ff3f861 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:13.187879 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399aac01-b223-4318-85fc-9b831ff3f861 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:13.188166 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399aac01-b223-4318-85fc-9b831ff3f861 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:13.189731 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-399aac01-b223-4318-85fc-9b831ff3f861 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:13.280034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399aac01-b223-4318-85fc-9b831ff3f861 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:13.281670 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399aac01-b223-4318-85fc-9b831ff3f861 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:13.300055 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 704/1422] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:14.180288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a2bf3-e83d-4694-bcd1-398468804286 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:14.180831 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a2bf3-e83d-4694-bcd1-398468804286 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:14.180958 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a2bf3-e83d-4694-bcd1-398468804286 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:14.182200 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:14.187201 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794ed14-e153-4a4d-95f8-4b995a903cf0 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:14.187592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794ed14-e153-4a4d-95f8-4b995a903cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:14.187751 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794ed14-e153-4a4d-95f8-4b995a903cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:14.208027 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:14.265438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-5794ed14-e153-4a4d-95f8-4b995a903cf0 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:14.266637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5794ed14-e153-4a4d-95f8-4b995a903cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:14.363649 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:14.514107 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a2bf3-e83d-4694-bcd1-398468804286 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:14.515504 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ee9a2bf3-e83d-4694-bcd1-398468804286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:14.626138 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:14.664668 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a2bf3-e83d-4694-bcd1-398468804286 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:14.777609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4f443-1be2-4000-b2dd-cf0c3a935cae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:14.778145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4f443-1be2-4000-b2dd-cf0c3a935cae service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:14.778424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4f443-1be2-4000-b2dd-cf0c3a935cae service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:14.778699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4f443-1be2-4000-b2dd-cf0c3a935cae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:14.780319 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3fc4f443-1be2-4000-b2dd-cf0c3a935cae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:14.856222 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:14.870452 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4f443-1be2-4000-b2dd-cf0c3a935cae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:14.872060 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4f443-1be2-4000-b2dd-cf0c3a935cae service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:14.890689 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 720/1425] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:15.751192 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6761e-30aa-4506-b02c-d4ade099684c None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:15.751663 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6761e-30aa-4506-b02c-d4ade099684c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:15.751773 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6761e-30aa-4506-b02c-d4ade099684c None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:15.753154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:15.758575 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3190ed-6f3d-4de2-983e-717c63d36c15 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:15.759029 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3190ed-6f3d-4de2-983e-717c63d36c15 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:15.759142 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3190ed-6f3d-4de2-983e-717c63d36c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:15.836350 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7b3190ed-6f3d-4de2-983e-717c63d36c15 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:15.837560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7b3190ed-6f3d-4de2-983e-717c63d36c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:15.935871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:49:16.088084 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6761e-30aa-4506-b02c-d4ade099684c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:16.089780 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c3d6761e-30aa-4506-b02c-d4ade099684c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:16.175063 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:16.237885 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6761e-30aa-4506-b02c-d4ade099684c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:16.355883 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40affdad-b062-4e2f-befa-894bed126d5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:16.356347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40affdad-b062-4e2f-befa-894bed126d5e service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:16.356730 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40affdad-b062-4e2f-befa-894bed126d5e service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:16.357140 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40affdad-b062-4e2f-befa-894bed126d5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:16.359830 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-40affdad-b062-4e2f-befa-894bed126d5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:16.413884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:16.455773 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40affdad-b062-4e2f-befa-894bed126d5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:16.457421 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40affdad-b062-4e2f-befa-894bed126d5e service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:16.476456 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 707/1428] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:17.281592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12117316-9a25-46d3-9c90-62916457f126 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:17.282200 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12117316-9a25-46d3-9c90-62916457f126 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:17.282200 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12117316-9a25-46d3-9c90-62916457f126 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:17.283475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:17.288535 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29f382-191b-4f70-b957-ca415a71c54b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:17.288994 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29f382-191b-4f70-b957-ca415a71c54b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:17.289098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29f382-191b-4f70-b957-ca415a71c54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:17.367088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-3f29f382-191b-4f70-b957-ca415a71c54b None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:17.368386 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3f29f382-191b-4f70-b957-ca415a71c54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:17.468038 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:17.622316 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12117316-9a25-46d3-9c90-62916457f126 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:17.623772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-12117316-9a25-46d3-9c90-62916457f126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:17.675128 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:17.777627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12117316-9a25-46d3-9c90-62916457f126 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:17.891962 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748dd81-4560-4246-8e69-b7b3ac7f3c15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:17.892600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748dd81-4560-4246-8e69-b7b3ac7f3c15 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:17.892879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748dd81-4560-4246-8e69-b7b3ac7f3c15 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:17.893150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748dd81-4560-4246-8e69-b7b3ac7f3c15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:17.894756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9748dd81-4560-4246-8e69-b7b3ac7f3c15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:17.903044 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:17.986987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748dd81-4560-4246-8e69-b7b3ac7f3c15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:17.988629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748dd81-4560-4246-8e69-b7b3ac7f3c15 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:18.006669 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 723/1431] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:18.826794 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c029d0-ce76-4904-a354-a793cab17731 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:18.827353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c029d0-ce76-4904-a354-a793cab17731 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:18.827353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c029d0-ce76-4904-a354-a793cab17731 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:18.828990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:18.834197 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f9861-87e0-4801-91a8-dc166b461cea None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:18.834625 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f9861-87e0-4801-91a8-dc166b461cea None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:18.834729 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f9861-87e0-4801-91a8-dc166b461cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:18.909252 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-bf3f9861-87e0-4801-91a8-dc166b461cea None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:18.910450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bf3f9861-87e0-4801-91a8-dc166b461cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:18.977685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:19.008520 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:19.159992 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c029d0-ce76-4904-a354-a793cab17731 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:19.161552 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-00c029d0-ce76-4904-a354-a793cab17731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:19.180881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:19.312085 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c029d0-ce76-4904-a354-a793cab17731 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:19.399541 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:19.426822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c02c4-8384-4789-ab61-66799ae41a45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:19.427431 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c02c4-8384-4789-ab61-66799ae41a45 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:19.427907 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c02c4-8384-4789-ab61-66799ae41a45 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:19.428361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c02c4-8384-4789-ab61-66799ae41a45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:19.429968 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dd3c02c4-8384-4789-ab61-66799ae41a45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:19.518727 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c02c4-8384-4789-ab61-66799ae41a45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:19.520391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c02c4-8384-4789-ab61-66799ae41a45 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:19.539405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 710/1434] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:20.350903 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95fd0a-7497-4241-a3b3-bd0c4de20c31 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:20.351475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95fd0a-7497-4241-a3b3-bd0c4de20c31 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:20.351475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95fd0a-7497-4241-a3b3-bd0c4de20c31 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:20.352862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:20.358103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e514150-54c0-4016-a17f-aeccbf68be45 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:20.358519 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e514150-54c0-4016-a17f-aeccbf68be45 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:20.358645 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e514150-54c0-4016-a17f-aeccbf68be45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:20.437084 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7e514150-54c0-4016-a17f-aeccbf68be45 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:20.438291 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7e514150-54c0-4016-a17f-aeccbf68be45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:20.479790 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:20.534411 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:20.685231 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95fd0a-7497-4241-a3b3-bd0c4de20c31 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:20.686712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cf95fd0a-7497-4241-a3b3-bd0c4de20c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:20.835970 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95fd0a-7497-4241-a3b3-bd0c4de20c31 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:20.890254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:20.949921 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c42261-30ff-4a16-8a82-7efc4c224262 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:20.950447 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c42261-30ff-4a16-8a82-7efc4c224262 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:20.950724 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c42261-30ff-4a16-8a82-7efc4c224262 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:20.950994 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c42261-30ff-4a16-8a82-7efc4c224262 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:20.952608 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-16c42261-30ff-4a16-8a82-7efc4c224262 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:21.041934 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c42261-30ff-4a16-8a82-7efc4c224262 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:21.043537 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c42261-30ff-4a16-8a82-7efc4c224262 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:21.062051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 726/1437] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:21.874485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdd6bb-24b4-4110-9e79-dc8d4205555f None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:21.875047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdd6bb-24b4-4110-9e79-dc8d4205555f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:21.875047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdd6bb-24b4-4110-9e79-dc8d4205555f None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:21.876438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:21.882199 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ac06a-f72b-4358-974b-f85b14c3f144 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:21.882664 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ac06a-f72b-4358-974b-f85b14c3f144 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:21.882810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ac06a-f72b-4358-974b-f85b14c3f144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:21.957386 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d33ac06a-f72b-4358-974b-f85b14c3f144 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:21.958576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d33ac06a-f72b-4358-974b-f85b14c3f144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:21.960721 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:22.055073 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:49:22.219294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdd6bb-24b4-4110-9e79-dc8d4205555f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:22.220887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-73bdd6bb-24b4-4110-9e79-dc8d4205555f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:22.369651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdd6bb-24b4-4110-9e79-dc8d4205555f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:22.398163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:22.485369 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba311cb5-10a0-45b4-aca9-fab66de73377 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:22.485995 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba311cb5-10a0-45b4-aca9-fab66de73377 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:22.486278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba311cb5-10a0-45b4-aca9-fab66de73377 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:22.486551 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba311cb5-10a0-45b4-aca9-fab66de73377 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:22.488212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ba311cb5-10a0-45b4-aca9-fab66de73377 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:22.576028 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba311cb5-10a0-45b4-aca9-fab66de73377 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:22.577662 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba311cb5-10a0-45b4-aca9-fab66de73377 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:22.596410 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 713/1440] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:24.404525 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a4412-bc3b-41bd-a1a4-3d2f59e6e935 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:24.405071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a4412-bc3b-41bd-a1a4-3d2f59e6e935 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:24.405071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a4412-bc3b-41bd-a1a4-3d2f59e6e935 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:24.406403 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:24.411638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b982eb0-0308-44fa-ade5-bb1d51992a61 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:24.412079 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b982eb0-0308-44fa-ade5-bb1d51992a61 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:24.412235 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b982eb0-0308-44fa-ade5-bb1d51992a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:24.457376 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:24.485038 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-4b982eb0-0308-44fa-ade5-bb1d51992a61 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:24.486236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4b982eb0-0308-44fa-ade5-bb1d51992a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:24.587058 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:24.873614 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a4412-bc3b-41bd-a1a4-3d2f59e6e935 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:24.875134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-643a4412-bc3b-41bd-a1a4-3d2f59e6e935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:24.978507 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a4412-bc3b-41bd-a1a4-3d2f59e6e935 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:25.010105 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:25.051949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d2993-297d-4c5d-8a1b-9162f452723a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:25.052588 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d2993-297d-4c5d-8a1b-9162f452723a service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:25.052869 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d2993-297d-4c5d-8a1b-9162f452723a service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:25.053810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d2993-297d-4c5d-8a1b-9162f452723a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:25.055469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f22d2993-297d-4c5d-8a1b-9162f452723a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:25.168279 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d2993-297d-4c5d-8a1b-9162f452723a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:25.171416 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d2993-297d-4c5d-8a1b-9162f452723a service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:25.190262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 729/1443] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:27.260830 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6fdbd-2864-4a18-ae63-ecb1f238cd5d None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:27.261448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6fdbd-2864-4a18-ae63-ecb1f238cd5d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:27.261448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6fdbd-2864-4a18-ae63-ecb1f238cd5d None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:27.262708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:27.268285 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe025f89-41ce-4e7c-96bd-52fdfb175c56 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:27.268679 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe025f89-41ce-4e7c-96bd-52fdfb175c56 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:27.268848 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe025f89-41ce-4e7c-96bd-52fdfb175c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:27.336887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-fe025f89-41ce-4e7c-96bd-52fdfb175c56 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:27.338068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fe025f89-41ce-4e7c-96bd-52fdfb175c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:27.351054 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:27.441793 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:27.726775 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6fdbd-2864-4a18-ae63-ecb1f238cd5d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:27.728384 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-26f6fdbd-2864-4a18-ae63-ecb1f238cd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:27.831794 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6fdbd-2864-4a18-ae63-ecb1f238cd5d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:27.843888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:27.907970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e666d2-dfd0-4ee2-a99a-d4a812f4cf70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:27.908610 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e666d2-dfd0-4ee2-a99a-d4a812f4cf70 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:27.908888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e666d2-dfd0-4ee2-a99a-d4a812f4cf70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:27.909157 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e666d2-dfd0-4ee2-a99a-d4a812f4cf70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:27.910747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f6e666d2-dfd0-4ee2-a99a-d4a812f4cf70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:28.020383 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e666d2-dfd0-4ee2-a99a-d4a812f4cf70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:28.021759 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e666d2-dfd0-4ee2-a99a-d4a812f4cf70 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:28.040157 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 716/1446] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:30.302682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7512853-208d-4b60-849b-53f0a09aca4d None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:30.303353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7512853-208d-4b60-849b-53f0a09aca4d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:30.303353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7512853-208d-4b60-849b-53f0a09aca4d None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:30.305180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:30.310513 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa84e9e-6996-451c-be0f-f69b272aaf21 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:30.310918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa84e9e-6996-451c-be0f-f69b272aaf21 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:30.311068 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa84e9e-6996-451c-be0f-f69b272aaf21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:30.360292 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:30.378985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-2fa84e9e-6996-451c-be0f-f69b272aaf21 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:30.380242 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2fa84e9e-6996-451c-be0f-f69b272aaf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:30.478000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:30.767456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7512853-208d-4b60-849b-53f0a09aca4d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:30.769016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e7512853-208d-4b60-849b-53f0a09aca4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:30.876016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7512853-208d-4b60-849b-53f0a09aca4d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:30.918460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:30.954019 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e74b8-641b-46c6-927c-97bf7479d6b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:30.954625 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e74b8-641b-46c6-927c-97bf7479d6b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:30.954916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e74b8-641b-46c6-927c-97bf7479d6b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:30.955164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e74b8-641b-46c6-927c-97bf7479d6b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:30.956769 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-762e74b8-641b-46c6-927c-97bf7479d6b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:31.065365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e74b8-641b-46c6-927c-97bf7479d6b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:31.066709 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e74b8-641b-46c6-927c-97bf7479d6b0 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:31.085646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 732/1449] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:33.145509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef1e29-7d34-497f-86ec-eafbbf38fe08 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:33.146092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef1e29-7d34-497f-86ec-eafbbf38fe08 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:33.146092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef1e29-7d34-497f-86ec-eafbbf38fe08 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:33.148116 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:33.154004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a2fb3-8c40-492a-b276-345fb55f62c3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:33.154502 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a2fb3-8c40-492a-b276-345fb55f62c3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:33.154616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a2fb3-8c40-492a-b276-345fb55f62c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:33.221747 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f26a2fb3-8c40-492a-b276-345fb55f62c3 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:33.222953 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f26a2fb3-8c40-492a-b276-345fb55f62c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:33.243999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:33.325507 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:33.609825 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef1e29-7d34-497f-86ec-eafbbf38fe08 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:33.611348 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-45ef1e29-7d34-497f-86ec-eafbbf38fe08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:33.719272 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef1e29-7d34-497f-86ec-eafbbf38fe08 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:33.739432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:33.793970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35f07c-d1ec-4b06-9755-183dfcfe7882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:33.794580 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35f07c-d1ec-4b06-9755-183dfcfe7882 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:33.794862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35f07c-d1ec-4b06-9755-183dfcfe7882 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:33.795134 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35f07c-d1ec-4b06-9755-183dfcfe7882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:33.796856 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0d35f07c-d1ec-4b06-9755-183dfcfe7882 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:33.906820 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35f07c-d1ec-4b06-9755-183dfcfe7882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:33.908293 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35f07c-d1ec-4b06-9755-183dfcfe7882 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:33.926583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 719/1452] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:35.905917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22baf42b-1aa4-489c-99e9-f4378a390494 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:35.906414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22baf42b-1aa4-489c-99e9-f4378a390494 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:35.906512 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22baf42b-1aa4-489c-99e9-f4378a390494 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:35.907865 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:35.913573 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae441cea-0f27-418c-924e-331ee8048bed None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:35.913970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae441cea-0f27-418c-924e-331ee8048bed None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:35.914133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae441cea-0f27-418c-924e-331ee8048bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:35.983504 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:35.986804 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-ae441cea-0f27-418c-924e-331ee8048bed None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:35.988517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ae441cea-0f27-418c-924e-331ee8048bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:36.089987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:36.382807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22baf42b-1aa4-489c-99e9-f4378a390494 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:36.384404 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-22baf42b-1aa4-489c-99e9-f4378a390494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:36.489715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22baf42b-1aa4-489c-99e9-f4378a390494 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:36.539662 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:36.564767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f687e-4cf2-4cbe-abef-917242345043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:36.565375 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f687e-4cf2-4cbe-abef-917242345043 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:36.565649 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f687e-4cf2-4cbe-abef-917242345043 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:36.565918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f687e-4cf2-4cbe-abef-917242345043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:36.567495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5e7f687e-4cf2-4cbe-abef-917242345043 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:36.678247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f687e-4cf2-4cbe-abef-917242345043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:36.679872 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f687e-4cf2-4cbe-abef-917242345043 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:36.697568 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 735/1455] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:38.699565 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed334658-b11f-4cae-aa4a-975679cd797b None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:38.700132 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed334658-b11f-4cae-aa4a-975679cd797b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:38.700132 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed334658-b11f-4cae-aa4a-975679cd797b None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:38.701515 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:38.707175 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b916c-a241-4e52-9069-f6ee41e94acf None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:38.707598 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b916c-a241-4e52-9069-f6ee41e94acf None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:38.707727 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b916c-a241-4e52-9069-f6ee41e94acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:38.775418 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7a2b916c-a241-4e52-9069-f6ee41e94acf None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:38.776646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7a2b916c-a241-4e52-9069-f6ee41e94acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:38.805782 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:38.875193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:39.170288 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed334658-b11f-4cae-aa4a-975679cd797b None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:39.171810 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ed334658-b11f-4cae-aa4a-975679cd797b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:39.275229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed334658-b11f-4cae-aa4a-975679cd797b None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:39.303581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:39.349089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546bbb6-a17f-4bd0-a10e-86281a57e73a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:39.349699 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546bbb6-a17f-4bd0-a10e-86281a57e73a service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:39.349981 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546bbb6-a17f-4bd0-a10e-86281a57e73a service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:39.350253 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546bbb6-a17f-4bd0-a10e-86281a57e73a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:39.352565 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c546bbb6-a17f-4bd0-a10e-86281a57e73a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:39.462660 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546bbb6-a17f-4bd0-a10e-86281a57e73a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:39.463904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546bbb6-a17f-4bd0-a10e-86281a57e73a service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:39.482753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 722/1458] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:41.560002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134b0dc-316f-471c-886d-e38bb7bfc70c None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:41.560645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134b0dc-316f-471c-886d-e38bb7bfc70c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:41.560645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134b0dc-316f-471c-886d-e38bb7bfc70c None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:41.562876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:41.569312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb8b63-e3e7-4ab4-a840-38440e681234 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:41.569843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb8b63-e3e7-4ab4-a840-38440e681234 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:41.569843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb8b63-e3e7-4ab4-a840-38440e681234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:41.652289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-96eb8b63-e3e7-4ab4-a840-38440e681234 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:41.653508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-96eb8b63-e3e7-4ab4-a840-38440e681234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:41.662609 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:41.763412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:42.056675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134b0dc-316f-471c-886d-e38bb7bfc70c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:42.058235 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e134b0dc-316f-471c-886d-e38bb7bfc70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:42.175554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134b0dc-316f-471c-886d-e38bb7bfc70c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:42.234787 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:42.253956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e7903-cb3d-4bd6-8285-0039c34702cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:42.254572 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e7903-cb3d-4bd6-8285-0039c34702cb service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:42.254852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e7903-cb3d-4bd6-8285-0039c34702cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:42.255124 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e7903-cb3d-4bd6-8285-0039c34702cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:42.256756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-661e7903-cb3d-4bd6-8285-0039c34702cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:42.376305 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e7903-cb3d-4bd6-8285-0039c34702cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:42.377747 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e7903-cb3d-4bd6-8285-0039c34702cb service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:42.402783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 738/1461] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:44.423302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc292e-a838-4a77-8390-3fe0a737d7ab None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:44.423896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc292e-a838-4a77-8390-3fe0a737d7ab None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:44.423896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc292e-a838-4a77-8390-3fe0a737d7ab None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:44.425245 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:44.431590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4d9af-d3c1-4e62-85b9-7ea84a424dfe None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:44.432076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4d9af-d3c1-4e62-85b9-7ea84a424dfe None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:44.432213 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4d9af-d3c1-4e62-85b9-7ea84a424dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:44.503393 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-13c4d9af-d3c1-4e62-85b9-7ea84a424dfe None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:44.504531 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-13c4d9af-d3c1-4e62-85b9-7ea84a424dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:44.543722 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:44.611041 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:44.903560 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc292e-a838-4a77-8390-3fe0a737d7ab None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:44.905162 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ebfc292e-a838-4a77-8390-3fe0a737d7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:45.013588 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc292e-a838-4a77-8390-3fe0a737d7ab None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:45.053940 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:45.091657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdcba1-64c5-48b4-9b41-67e7b2a58423 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:45.092291 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdcba1-64c5-48b4-9b41-67e7b2a58423 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:45.092575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdcba1-64c5-48b4-9b41-67e7b2a58423 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:45.092846 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdcba1-64c5-48b4-9b41-67e7b2a58423 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:45.094447 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0fcdcba1-64c5-48b4-9b41-67e7b2a58423 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:45.207023 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdcba1-64c5-48b4-9b41-67e7b2a58423 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:45.208500 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdcba1-64c5-48b4-9b41-67e7b2a58423 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:45.229052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 725/1464] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:47.251199 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f926e-028c-44d9-b6ef-c86f3ec83837 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:47.251747 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f926e-028c-44d9-b6ef-c86f3ec83837 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:47.251747 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f926e-028c-44d9-b6ef-c86f3ec83837 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:47.253158 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:47.259161 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675ccc7-2785-440c-9fcc-2ebaa7468aa7 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:47.259562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675ccc7-2785-440c-9fcc-2ebaa7468aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:47.259712 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675ccc7-2785-440c-9fcc-2ebaa7468aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:47.331920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-5675ccc7-2785-440c-9fcc-2ebaa7468aa7 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:47.333160 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5675ccc7-2785-440c-9fcc-2ebaa7468aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:47.356689 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:47.441372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:47.727832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f926e-028c-44d9-b6ef-c86f3ec83837 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:47.729392 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6c2f926e-028c-44d9-b6ef-c86f3ec83837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:47.859483 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f926e-028c-44d9-b6ef-c86f3ec83837 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:47.947567 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:47.956863 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d11b8-a04d-40f5-a0c9-9e4f36ee629f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:47.957509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d11b8-a04d-40f5-a0c9-9e4f36ee629f service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:47.957817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d11b8-a04d-40f5-a0c9-9e4f36ee629f service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:47.958093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d11b8-a04d-40f5-a0c9-9e4f36ee629f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:47.959950 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-784d11b8-a04d-40f5-a0c9-9e4f36ee629f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:48.094486 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d11b8-a04d-40f5-a0c9-9e4f36ee629f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:48.095975 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d11b8-a04d-40f5-a0c9-9e4f36ee629f service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:48.119308 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 741/1467] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:49:50.140057 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40010eb-2c75-4cbb-a802-3a2eafe17bef None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:50.140657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40010eb-2c75-4cbb-a802-3a2eafe17bef None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:50.140657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40010eb-2c75-4cbb-a802-3a2eafe17bef None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:50.142316 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:49:50.147825 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0812b-8612-4f0c-9626-62c5911d6469 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:50.148946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0812b-8612-4f0c-9626-62c5911d6469 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:50.149091 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0812b-8612-4f0c-9626-62c5911d6469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:50.218729 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0eb0812b-8612-4f0c-9626-62c5911d6469 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:49:50.219943 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0eb0812b-8612-4f0c-9626-62c5911d6469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:50.281462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:50.331086 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:49:50.619664 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40010eb-2c75-4cbb-a802-3a2eafe17bef None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:49:50.621248 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b40010eb-2c75-4cbb-a802-3a2eafe17bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:50.731618 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40010eb-2c75-4cbb-a802-3a2eafe17bef None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:49:50.782217 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:49:50.810107 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba5c96-a5ee-4b6f-b3c9-9ca71d827b83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:49:50.810720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba5c96-a5ee-4b6f-b3c9-9ca71d827b83 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:49:50.811000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba5c96-a5ee-4b6f-b3c9-9ca71d827b83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:49:50.811270 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba5c96-a5ee-4b6f-b3c9-9ca71d827b83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:49:50.812922 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d6ba5c96-a5ee-4b6f-b3c9-9ca71d827b83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:49:50.928898 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba5c96-a5ee-4b6f-b3c9-9ca71d827b83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:49:50.930447 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba5c96-a5ee-4b6f-b3c9-9ca71d827b83 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:49:50.950034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 728/1470] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:49:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:00.328901 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=22 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:00.947506 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=66 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:10.750635 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304118c2-b7fb-4682-8538-79599d2a4693 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:10.751156 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304118c2-b7fb-4682-8538-79599d2a4693 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:10.751264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304118c2-b7fb-4682-8538-79599d2a4693 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:10.752649 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 743/1471] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:10.758287 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1a8a0-0407-413d-be2c-ecb5718e3080 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:10.758868 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1a8a0-0407-413d-be2c-ecb5718e3080 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:10.758868 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1a8a0-0407-413d-be2c-ecb5718e3080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:10.826141 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-a3c1a8a0-0407-413d-be2c-ecb5718e3080 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:10.827335 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a3c1a8a0-0407-413d-be2c-ecb5718e3080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:10.924642 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 729/1472] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:50:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:11.220648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4072df-6332-4886-8701-af474a946337 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:11.220934 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4072df-6332-4886-8701-af474a946337 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:11.221212 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4072df-6332-4886-8701-af474a946337 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:11.222566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 744/1473] 10.222.0.67 () {54 vars in 967 bytes} [Wed Aug 24 08:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:11.228372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03ed77-c280-495d-889f-742365f80c45 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:11.228669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03ed77-c280-495d-889f-742365f80c45 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:11.228944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03ed77-c280-495d-889f-742365f80c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:11.308334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-ba03ed77-c280-495d-889f-742365f80c45 None None] MFA Rules not processed for user `5979387f7aa54619b8d429368c306cc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:11.309540 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ba03ed77-c280-495d-889f-742365f80c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:11.329562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:11.387830 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 730/1474] 10.222.0.67 () {58 vars in 1064 bytes} [Wed Aug 24 08:50:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:11.395530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4072df-6332-4886-8701-af474a946337 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:11.397096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8b4072df-6332-4886-8701-af474a946337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:11.515872 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4072df-6332-4886-8701-af474a946337 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:11.590728 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc3bd2-02e1-499d-8223-095fc992ea42 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5979387f7aa54619b8d429368c306cc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:11.591337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc3bd2-02e1-499d-8223-095fc992ea42 service glance] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:11.591616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc3bd2-02e1-499d-8223-095fc992ea42 service glance] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:11.591887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc3bd2-02e1-499d-8223-095fc992ea42 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:11.593501 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a4bc3bd2-02e1-499d-8223-095fc992ea42 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:11.651223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:11.706329 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc3bd2-02e1-499d-8223-095fc992ea42 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:11.708115 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc3bd2-02e1-499d-8223-095fc992ea42 service glance] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:11.727033 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 745/1475] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:13.756895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2db54d-81b6-461f-a214-db378f88b4bc None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:13.757401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2db54d-81b6-461f-a214-db378f88b4bc None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:13.757520 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2db54d-81b6-461f-a214-db378f88b4bc None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:13.759404 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 731/1476] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:50:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:13.764830 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f563143-a4ed-4f87-bbfe-fcb52629d0aa None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:13.765279 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f563143-a4ed-4f87-bbfe-fcb52629d0aa None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:13.765378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f563143-a4ed-4f87-bbfe-fcb52629d0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:13.832540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2f563143-a4ed-4f87-bbfe-fcb52629d0aa None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:13.833757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2f563143-a4ed-4f87-bbfe-fcb52629d0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:13.931837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 746/1477] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:14.206996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2db54d-81b6-461f-a214-db378f88b4bc None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:14.208971 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-bb2db54d-81b6-461f-a214-db378f88b4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:14.334790 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2db54d-81b6-461f-a214-db378f88b4bc None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:14.392395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:14.439206 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2200a-b4b1-4c1b-8922-ad92a86ef006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:14.439816 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2200a-b4b1-4c1b-8922-ad92a86ef006 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:14.440113 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2200a-b4b1-4c1b-8922-ad92a86ef006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:14.440392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2200a-b4b1-4c1b-8922-ad92a86ef006 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:14.441030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2200a-b4b1-4c1b-8922-ad92a86ef006 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:14.442793 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: WARNING py.warnings [None req-22c2200a-b4b1-4c1b-8922-ad92a86ef006 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:50:14.442793 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: warnings.warn(msg) Aug 24 08:50:14.442793 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: Aug 24 08:50:14.443939 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2200a-b4b1-4c1b-8922-ad92a86ef006 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:14.464693 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 732/1478] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 24 08:50:14 2022] GET /identity/v3/projects => generated 2553 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:50:16.462762 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a795e-c8d8-49a9-bc04-e24a19f71e0f None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:16.463310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a795e-c8d8-49a9-bc04-e24a19f71e0f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:16.463310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a795e-c8d8-49a9-bc04-e24a19f71e0f None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:16.464689 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 747/1479] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:16.470097 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd556d-0bd3-401e-9b91-26ca7f39b82f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:16.470560 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd556d-0bd3-401e-9b91-26ca7f39b82f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:16.470710 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd556d-0bd3-401e-9b91-26ca7f39b82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:16.538295 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-c8bd556d-0bd3-401e-9b91-26ca7f39b82f None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:16.539506 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c8bd556d-0bd3-401e-9b91-26ca7f39b82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:16.638441 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 733/1480] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:16.947276 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a795e-c8d8-49a9-bc04-e24a19f71e0f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:16.948728 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-294a795e-c8d8-49a9-bc04-e24a19f71e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:16.958959 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:17.225398 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a795e-c8d8-49a9-bc04-e24a19f71e0f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:17.307232 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:17.349456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e761f-abeb-47f6-9504-f1d48badc453 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:17.350076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e761f-abeb-47f6-9504-f1d48badc453 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:17.350348 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e761f-abeb-47f6-9504-f1d48badc453 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:17.350612 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e761f-abeb-47f6-9504-f1d48badc453 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:17.352260 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ba9e761f-abeb-47f6-9504-f1d48badc453 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:17.450709 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e761f-abeb-47f6-9504-f1d48badc453 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:17.452323 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e761f-abeb-47f6-9504-f1d48badc453 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:17.471290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 748/1481] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:19.607894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43db2c-674c-477d-bc3b-cd1b8c90d15b None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:19.608590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43db2c-674c-477d-bc3b-cd1b8c90d15b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:19.608590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43db2c-674c-477d-bc3b-cd1b8c90d15b None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:19.609815 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 734/1482] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:19.615181 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb856b8-ae6f-46e5-b51b-740bfdaeb531 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:19.615671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb856b8-ae6f-46e5-b51b-740bfdaeb531 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:19.615781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb856b8-ae6f-46e5-b51b-740bfdaeb531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:19.691818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2bb856b8-ae6f-46e5-b51b-740bfdaeb531 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:19.693058 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2bb856b8-ae6f-46e5-b51b-740bfdaeb531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:19.740838 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:19.796219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 749/1483] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:20.088896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43db2c-674c-477d-bc3b-cd1b8c90d15b None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:20.090533 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7d43db2c-674c-477d-bc3b-cd1b8c90d15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:20.119048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:20.202884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43db2c-674c-477d-bc3b-cd1b8c90d15b None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:20.281806 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c3802-682d-472b-bda9-c5bdc73bad26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:20.282557 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c3802-682d-472b-bda9-c5bdc73bad26 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:20.282938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c3802-682d-472b-bda9-c5bdc73bad26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:20.283309 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c3802-682d-472b-bda9-c5bdc73bad26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:20.285128 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e07c3802-682d-472b-bda9-c5bdc73bad26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:20.362038 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:20.399294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c3802-682d-472b-bda9-c5bdc73bad26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:20.401381 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c3802-682d-472b-bda9-c5bdc73bad26 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:20.420486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 735/1484] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:22.396006 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2f519-2b71-4d33-8a7f-83d95d29ff6a None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:22.396590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2f519-2b71-4d33-8a7f-83d95d29ff6a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:22.396590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2f519-2b71-4d33-8a7f-83d95d29ff6a None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:22.397929 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 750/1485] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:22.404437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c30d9-3fac-4b47-8ed7-d25eb5d41f84 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:22.404985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c30d9-3fac-4b47-8ed7-d25eb5d41f84 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:22.404985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c30d9-3fac-4b47-8ed7-d25eb5d41f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:22.473473 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-517c30d9-3fac-4b47-8ed7-d25eb5d41f84 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:22.474697 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-517c30d9-3fac-4b47-8ed7-d25eb5d41f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:22.572304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 736/1486] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:22.871948 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2f519-2b71-4d33-8a7f-83d95d29ff6a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:22.873485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f4e2f519-2b71-4d33-8a7f-83d95d29ff6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:22.982088 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2f519-2b71-4d33-8a7f-83d95d29ff6a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:23.054942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:23.056863 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf7abc-4389-4603-ac0e-99c5620b8907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:23.057487 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf7abc-4389-4603-ac0e-99c5620b8907 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:23.057952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf7abc-4389-4603-ac0e-99c5620b8907 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:23.058306 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf7abc-4389-4603-ac0e-99c5620b8907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:23.060375 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f4cf7abc-4389-4603-ac0e-99c5620b8907 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:23.173325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf7abc-4389-4603-ac0e-99c5620b8907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:23.174761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf7abc-4389-4603-ac0e-99c5620b8907 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:23.195227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 751/1487] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:25.367099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46206acd-e3a2-4322-9493-7326d8de3224 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:25.367724 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46206acd-e3a2-4322-9493-7326d8de3224 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:25.367724 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46206acd-e3a2-4322-9493-7326d8de3224 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:25.370143 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 737/1488] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:50:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:50:25.376602 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064236ff-28ef-4704-98df-3a5dd35d92f6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:25.377452 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064236ff-28ef-4704-98df-3a5dd35d92f6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:25.377452 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064236ff-28ef-4704-98df-3a5dd35d92f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:25.460644 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-064236ff-28ef-4704-98df-3a5dd35d92f6 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:25.461867 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-064236ff-28ef-4704-98df-3a5dd35d92f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:25.523405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:25.575606 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 752/1489] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:25.891657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46206acd-e3a2-4322-9493-7326d8de3224 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:25.894427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-46206acd-e3a2-4322-9493-7326d8de3224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:25.961350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:26.074712 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46206acd-e3a2-4322-9493-7326d8de3224 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:26.172217 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17027ba-964e-4ee8-bcd4-9b65c565f56f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:26.172551 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17027ba-964e-4ee8-bcd4-9b65c565f56f service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:26.172693 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17027ba-964e-4ee8-bcd4-9b65c565f56f service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:26.172877 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17027ba-964e-4ee8-bcd4-9b65c565f56f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:26.174519 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a17027ba-964e-4ee8-bcd4-9b65c565f56f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:26.274528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:26.309707 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17027ba-964e-4ee8-bcd4-9b65c565f56f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:26.311169 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17027ba-964e-4ee8-bcd4-9b65c565f56f service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:26.332640 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 738/1490] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:50:28.636898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438af65-2e8e-49b3-9ea9-f7f221f64fdf None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:28.637633 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438af65-2e8e-49b3-9ea9-f7f221f64fdf None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:28.637633 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438af65-2e8e-49b3-9ea9-f7f221f64fdf None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:28.638894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 753/1491] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 24 08:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:50:28.646226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3ca6f-76fc-4946-8d97-5dfcfe33e044 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:28.646840 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3ca6f-76fc-4946-8d97-5dfcfe33e044 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:28.647154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3ca6f-76fc-4946-8d97-5dfcfe33e044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:28.740381 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-89b3ca6f-76fc-4946-8d97-5dfcfe33e044 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:50:28.741816 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-89b3ca6f-76fc-4946-8d97-5dfcfe33e044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:28.866724 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 739/1492] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 24 08:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:50:29.169512 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438af65-2e8e-49b3-9ea9-f7f221f64fdf None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:50:29.171036 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4438af65-2e8e-49b3-9ea9-f7f221f64fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:29.296356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438af65-2e8e-49b3-9ea9-f7f221f64fdf None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:50:29.379295 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243f5e0-1c94-4611-8ea3-15dac963636f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:50:29.379909 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243f5e0-1c94-4611-8ea3-15dac963636f service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:50:29.380211 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243f5e0-1c94-4611-8ea3-15dac963636f service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:50:29.380472 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243f5e0-1c94-4611-8ea3-15dac963636f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:50:29.382131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7243f5e0-1c94-4611-8ea3-15dac963636f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:50:29.393726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:29.507803 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243f5e0-1c94-4611-8ea3-15dac963636f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:50:29.509167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243f5e0-1c94-4611-8ea3-15dac963636f service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:50:29.528982 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 754/1493] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:50:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:50:38.864248 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=82 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:50:39.526370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=50 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:50.992715 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3d19d-0c46-4e66-a584-c5fbae8afe6e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:50.993378 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3d19d-0c46-4e66-a584-c5fbae8afe6e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:50.993378 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3d19d-0c46-4e66-a584-c5fbae8afe6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:51.122819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-3cb3d19d-0c46-4e66-a584-c5fbae8afe6e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:51:51.124831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3cb3d19d-0c46-4e66-a584-c5fbae8afe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:51.243456 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 740/1494] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:51:51.252033 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8b52c-1343-49ea-80e8-c39283627370 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:51.252708 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8b52c-1343-49ea-80e8-c39283627370 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:51.252708 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8b52c-1343-49ea-80e8-c39283627370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:51.345209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-15f8b52c-1343-49ea-80e8-c39283627370 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:51:51.346640 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-15f8b52c-1343-49ea-80e8-c39283627370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:51.468957 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 755/1495] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:51:51.474059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3d19d-0c46-4e66-a584-c5fbae8afe6e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:51.475336 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3cb3d19d-0c46-4e66-a584-c5fbae8afe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:51.489386 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3d19d-0c46-4e66-a584-c5fbae8afe6e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:51.695927 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:51.743433 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d566cbc-f868-4cd3-969c-0c5479ad57b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:51.744293 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d566cbc-f868-4cd3-969c-0c5479ad57b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:51.744404 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d566cbc-f868-4cd3-969c-0c5479ad57b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:51.744621 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d566cbc-f868-4cd3-969c-0c5479ad57b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:51.745280 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d566cbc-f868-4cd3-969c-0c5479ad57b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:51:51.745581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d566cbc-f868-4cd3-969c-0c5479ad57b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:51.746620 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: WARNING py.warnings [None req-9d566cbc-f868-4cd3-969c-0c5479ad57b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:51:51.746620 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: warnings.warn(msg) Aug 24 08:51:51.746620 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: Aug 24 08:51:51.747519 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d566cbc-f868-4cd3-969c-0c5479ad57b8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:51.762075 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 741/1496] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:51:51.772754 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8b52c-1343-49ea-80e8-c39283627370 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:51.788185 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8b52c-1343-49ea-80e8-c39283627370 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:51.910030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:51.954559 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a732175-218b-4a7f-94ab-91657a4b99d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:51.955179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a732175-218b-4a7f-94ab-91657a4b99d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:51.955456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a732175-218b-4a7f-94ab-91657a4b99d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:51.955724 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a732175-218b-4a7f-94ab-91657a4b99d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:51.956398 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a732175-218b-4a7f-94ab-91657a4b99d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:51.958206 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-5a732175-218b-4a7f-94ab-91657a4b99d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:51:51.958206 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Aug 24 08:51:51.958206 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: Aug 24 08:51:51.959420 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a732175-218b-4a7f-94ab-91657a4b99d3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:52.035749 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 756/1497] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:51:51 2022] POST /identity/v3/projects => generated 377 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:51:52.041122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d566cbc-f868-4cd3-969c-0c5479ad57b8 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:52.042651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9d566cbc-f868-4cd3-969c-0c5479ad57b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:52.094046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d566cbc-f868-4cd3-969c-0c5479ad57b8 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:52.223004 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe36532-8702-4386-8099-8552dfe710fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:52.223631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe36532-8702-4386-8099-8552dfe710fe admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:52.223907 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe36532-8702-4386-8099-8552dfe710fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:52.224207 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe36532-8702-4386-8099-8552dfe710fe admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:52.224855 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe36532-8702-4386-8099-8552dfe710fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:52.225876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: WARNING py.warnings [None req-fbe36532-8702-4386-8099-8552dfe710fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:51:52.225876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: warnings.warn(msg) Aug 24 08:51:52.225876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: Aug 24 08:51:52.227105 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe36532-8702-4386-8099-8552dfe710fe admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:52.282892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 742/1498] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:51:52 2022] POST /identity/v3/users => generated 289 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:51:52.287597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a732175-218b-4a7f-94ab-91657a4b99d3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:52.289091 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5a732175-218b-4a7f-94ab-91657a4b99d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:52.370680 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a732175-218b-4a7f-94ab-91657a4b99d3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:52.506116 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:52.530364 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e2523-2724-4e0f-bf5b-ebf2e13c9355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:52.531396 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e2523-2724-4e0f-bf5b-ebf2e13c9355 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:52.531886 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e2523-2724-4e0f-bf5b-ebf2e13c9355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:52.532406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e2523-2724-4e0f-bf5b-ebf2e13c9355 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:52.533599 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e2523-2724-4e0f-bf5b-ebf2e13c9355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:52.535485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-b91e2523-2724-4e0f-bf5b-ebf2e13c9355 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:51:52.535485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Aug 24 08:51:52.535485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: Aug 24 08:51:52.536811 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e2523-2724-4e0f-bf5b-ebf2e13c9355 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:52.562296 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 757/1499] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:51:52.566753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe36532-8702-4386-8099-8552dfe710fe admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:52.568522 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fbe36532-8702-4386-8099-8552dfe710fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:52.581947 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe36532-8702-4386-8099-8552dfe710fe admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:52.613872 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:52.705393 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1e87a-c82e-4516-8a5d-393adb478878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:52.706020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1e87a-c82e-4516-8a5d-393adb478878 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:52.706301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1e87a-c82e-4516-8a5d-393adb478878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:52.706573 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1e87a-c82e-4516-8a5d-393adb478878 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:52.707240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1e87a-c82e-4516-8a5d-393adb478878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:52.708325 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: WARNING py.warnings [None req-79a1e87a-c82e-4516-8a5d-393adb478878 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:51:52.708325 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: warnings.warn(msg) Aug 24 08:51:52.708325 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: Aug 24 08:51:52.709255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1e87a-c82e-4516-8a5d-393adb478878 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:52.725680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 743/1500] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:51:52.732629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e2523-2724-4e0f-bf5b-ebf2e13c9355 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:52.734402 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b91e2523-2724-4e0f-bf5b-ebf2e13c9355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:52.775176 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e2523-2724-4e0f-bf5b-ebf2e13c9355 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:52.893745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1c69e-9e21-4add-a3de-57d1a8a51a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:52.894424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1c69e-9e21-4add-a3de-57d1a8a51a90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:52.894705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1c69e-9e21-4add-a3de-57d1a8a51a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:52.894976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1c69e-9e21-4add-a3de-57d1a8a51a90 admin admin] PATH_INFO: `/v3/projects/a72a973b0eae437ea28f3e5104bb7b78/users/4c77c20d20ff432a99221a60962651de/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:52.922837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1c69e-9e21-4add-a3de-57d1a8a51a90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a72a973b0eae437ea28f3e5104bb7b78, user_id=4c77c20d20ff432a99221a60962651de, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:52.924146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-27e1c69e-9e21-4add-a3de-57d1a8a51a90 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:51:52.924146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Aug 24 08:51:52.924146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: Aug 24 08:51:52.926262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1c69e-9e21-4add-a3de-57d1a8a51a90 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:52.959533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 758/1501] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:51:52 2022] PUT /identity/v3/projects/a72a973b0eae437ea28f3e5104bb7b78/users/4c77c20d20ff432a99221a60962651de/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:51:52.980730 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0972b5-0e49-4f91-9eda-c6f117b18f99 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:52.981109 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0972b5-0e49-4f91-9eda-c6f117b18f99 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:52.981487 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0972b5-0e49-4f91-9eda-c6f117b18f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:53.061850 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-ac0972b5-0e49-4f91-9eda-c6f117b18f99 None None] MFA Rules not processed for user `4c77c20d20ff432a99221a60962651de`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:51:53.063275 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ac0972b5-0e49-4f91-9eda-c6f117b18f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:53.195034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 744/1502] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:51:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:51:53.209502 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0df0d0-bce1-42e2-9ac4-1fedd70bdae8 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:53.209781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0df0d0-bce1-42e2-9ac4-1fedd70bdae8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:53.210054 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0df0d0-bce1-42e2-9ac4-1fedd70bdae8 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:53.211432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 759/1503] 10.222.0.67 () {54 vars in 964 bytes} [Wed Aug 24 08:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:51:53.218258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6f750-9ede-4f23-8f5b-037022ab0976 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:53.218672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6f750-9ede-4f23-8f5b-037022ab0976 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:53.219047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6f750-9ede-4f23-8f5b-037022ab0976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:53.255884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:53.312346 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-adf6f750-9ede-4f23-8f5b-037022ab0976 None None] MFA Rules not processed for user `5de1586efc8c4dec898b0fb54b56acf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:51:53.313574 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-adf6f750-9ede-4f23-8f5b-037022ab0976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:53.416175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 745/1504] 10.222.0.67 () {58 vars in 1061 bytes} [Wed Aug 24 08:51:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:51:53.425094 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0df0d0-bce1-42e2-9ac4-1fedd70bdae8 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:53.426601 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6e0df0d0-bce1-42e2-9ac4-1fedd70bdae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:53.488630 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:53.550488 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0df0d0-bce1-42e2-9ac4-1fedd70bdae8 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:53.641027 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c445ff2-188a-4793-8419-496b670b80b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:53.641801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c445ff2-188a-4793-8419-496b670b80b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:53.642179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c445ff2-188a-4793-8419-496b670b80b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:53.642554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c445ff2-188a-4793-8419-496b670b80b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:53.644361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0c445ff2-188a-4793-8419-496b670b80b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:53.775421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c445ff2-188a-4793-8419-496b670b80b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:53.777183 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c445ff2-188a-4793-8419-496b670b80b6 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:53.782421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:53.801171 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 760/1505] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:51:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:51:53.854140 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348010fd-b872-41e4-b897-fccefba75681 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:53.854644 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348010fd-b872-41e4-b897-fccefba75681 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:53.855141 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348010fd-b872-41e4-b897-fccefba75681 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:53.857979 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 746/1506] 10.222.0.67 () {52 vars in 830 bytes} [Wed Aug 24 08:51:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:51:53.861850 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c445ff2-188a-4793-8419-496b670b80b6 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:53.863090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0c445ff2-188a-4793-8419-496b670b80b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:53.996928 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c445ff2-188a-4793-8419-496b670b80b6 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:54.127710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1a2de-7650-4b2d-8c87-1d6f667057d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:54.128405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1a2de-7650-4b2d-8c87-1d6f667057d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:54.128693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1a2de-7650-4b2d-8c87-1d6f667057d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:54.128979 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1a2de-7650-4b2d-8c87-1d6f667057d8 admin admin] PATH_INFO: `/v3/users/4c77c20d20ff432a99221a60962651de` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:54.144329 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1a2de-7650-4b2d-8c87-1d6f667057d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c77c20d20ff432a99221a60962651de)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:54.145380 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-08b1a2de-7650-4b2d-8c87-1d6f667057d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:51:54.145380 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Aug 24 08:51:54.145380 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: Aug 24 08:51:54.146631 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1a2de-7650-4b2d-8c87-1d6f667057d8 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:54.158922 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:54.228748 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-08b1a2de-7650-4b2d-8c87-1d6f667057d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c77c20d20ff432a99221a60962651de'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:54.229208 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-08b1a2de-7650-4b2d-8c87-1d6f667057d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c77c20d20ff432a99221a60962651de'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:54.240567 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-08b1a2de-7650-4b2d-8c87-1d6f667057d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c77c20d20ff432a99221a60962651de'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:54.261149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-08b1a2de-7650-4b2d-8c87-1d6f667057d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c77c20d20ff432a99221a60962651de'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:54.290082 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-08b1a2de-7650-4b2d-8c87-1d6f667057d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c77c20d20ff432a99221a60962651de'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:54.297235 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 761/1507] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:51:53 2022] DELETE /identity/v3/users/4c77c20d20ff432a99221a60962651de => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:51:54.302893 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547d5aa-07c9-4b4e-8784-2fdc419fcd53 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:54.303726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547d5aa-07c9-4b4e-8784-2fdc419fcd53 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:54.303929 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547d5aa-07c9-4b4e-8784-2fdc419fcd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:54.371831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-3547d5aa-07c9-4b4e-8784-2fdc419fcd53 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:51:54.373122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3547d5aa-07c9-4b4e-8784-2fdc419fcd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:54.487508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:54.502691 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 747/1508] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:51:54.512527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1a2de-7650-4b2d-8c87-1d6f667057d8 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:54.513725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-08b1a2de-7650-4b2d-8c87-1d6f667057d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:54.650710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1a2de-7650-4b2d-8c87-1d6f667057d8 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:54.734827 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:51:54.759063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607c23d-38bf-430e-964e-98562e993123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:54.759689 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607c23d-38bf-430e-964e-98562e993123 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:54.759965 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607c23d-38bf-430e-964e-98562e993123 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:54.760281 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607c23d-38bf-430e-964e-98562e993123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:54.761926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f607c23d-38bf-430e-964e-98562e993123 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:54.892781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607c23d-38bf-430e-964e-98562e993123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:54.894407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607c23d-38bf-430e-964e-98562e993123 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:54.915695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 762/1509] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:51:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:51:55.315350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547d5aa-07c9-4b4e-8784-2fdc419fcd53 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:51:55.317394 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3547d5aa-07c9-4b4e-8784-2fdc419fcd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:51:55.334033 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547d5aa-07c9-4b4e-8784-2fdc419fcd53 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:51:55.459749 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c683d64-3ccb-44ea-ad9f-f3a648b808a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:51:55.460344 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c683d64-3ccb-44ea-ad9f-f3a648b808a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:51:55.460620 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c683d64-3ccb-44ea-ad9f-f3a648b808a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:51:55.460886 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c683d64-3ccb-44ea-ad9f-f3a648b808a6 admin admin] PATH_INFO: `/v3/projects/a72a973b0eae437ea28f3e5104bb7b78` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:51:55.475201 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c683d64-3ccb-44ea-ad9f-f3a648b808a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a72a973b0eae437ea28f3e5104bb7b78)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:51:55.476269 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: WARNING py.warnings [None req-2c683d64-3ccb-44ea-ad9f-f3a648b808a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:51:55.476269 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: warnings.warn(msg) Aug 24 08:51:55.476269 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: Aug 24 08:51:55.477556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c683d64-3ccb-44ea-ad9f-f3a648b808a6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:51:55.547584 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2c683d64-3ccb-44ea-ad9f-f3a648b808a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a72a973b0eae437ea28f3e5104bb7b78', 'request_id': 'req-2c683d64-3ccb-44ea-ad9f-f3a648b808a6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:55.557790 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2c683d64-3ccb-44ea-ad9f-f3a648b808a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a72a973b0eae437ea28f3e5104bb7b78', 'request_id': 'req-2c683d64-3ccb-44ea-ad9f-f3a648b808a6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:55.565390 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2c683d64-3ccb-44ea-ad9f-f3a648b808a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a72a973b0eae437ea28f3e5104bb7b78', 'request_id': 'req-2c683d64-3ccb-44ea-ad9f-f3a648b808a6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:55.600859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2c683d64-3ccb-44ea-ad9f-f3a648b808a6 admin admin] The token cache is being invalidate because project a72a973b0eae437ea28f3e5104bb7b78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:51:55.601204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2c683d64-3ccb-44ea-ad9f-f3a648b808a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:55.601621 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2c683d64-3ccb-44ea-ad9f-f3a648b808a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:51:55.603090 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 748/1510] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:51:55 2022] DELETE /identity/v3/projects/a72a973b0eae437ea28f3e5104bb7b78 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:52:04.913196 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=62 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:52:05.598922 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:53:27.450105 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd03f50-b3b1-475a-95b7-d245a97bcf9e None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:27.450932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd03f50-b3b1-475a-95b7-d245a97bcf9e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:27.451296 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd03f50-b3b1-475a-95b7-d245a97bcf9e None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:27.454961 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:53:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:53:27.460934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b146f-357c-4e7d-8f5d-a54f72e6c025 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:27.461684 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b146f-357c-4e7d-8f5d-a54f72e6c025 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:27.461684 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b146f-357c-4e7d-8f5d-a54f72e6c025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:27.580508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-ba1b146f-357c-4e7d-8f5d-a54f72e6c025 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:27.581743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ba1b146f-357c-4e7d-8f5d-a54f72e6c025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:27.719063 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:53:27.887415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd03f50-b3b1-475a-95b7-d245a97bcf9e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:53:27.892000 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bbd03f50-b3b1-475a-95b7-d245a97bcf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:28.113053 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd03f50-b3b1-475a-95b7-d245a97bcf9e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:53:28.122555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51884024-6563-49b8-9a30-a9304414eeda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:53:28.123344 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51884024-6563-49b8-9a30-a9304414eeda service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:28.123722 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51884024-6563-49b8-9a30-a9304414eeda service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:28.124221 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51884024-6563-49b8-9a30-a9304414eeda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:28.141364 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51884024-6563-49b8-9a30-a9304414eeda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:53:28.144611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51884024-6563-49b8-9a30-a9304414eeda service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:53:28.159326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 764/1513] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:53:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:53:29.272263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bb302-a4de-4d72-8a9e-4291eae7026e None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:29.273127 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bb302-a4de-4d72-8a9e-4291eae7026e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:29.273127 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bb302-a4de-4d72-8a9e-4291eae7026e None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:29.274449 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:53:29.281445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf7588-6afd-414d-8179-54f374ee12e3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:29.282092 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf7588-6afd-414d-8179-54f374ee12e3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:29.282254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf7588-6afd-414d-8179-54f374ee12e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:29.306781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-fabf7588-6afd-414d-8179-54f374ee12e3 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:29.308220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fabf7588-6afd-414d-8179-54f374ee12e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:29.318585 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:53:29.547728 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bb302-a4de-4d72-8a9e-4291eae7026e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:53:29.573888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bb302-a4de-4d72-8a9e-4291eae7026e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:53:29.587103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb8a86-cda1-4373-a79d-2c994fcd7032 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:53:29.587796 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb8a86-cda1-4373-a79d-2c994fcd7032 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:29.588082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb8a86-cda1-4373-a79d-2c994fcd7032 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:29.588385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb8a86-cda1-4373-a79d-2c994fcd7032 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:29.598746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb8a86-cda1-4373-a79d-2c994fcd7032 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:53:29.600534 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb8a86-cda1-4373-a79d-2c994fcd7032 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:53:29.611466 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 751/1516] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:53:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:53:29.685778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a592f-870b-4252-92c6-5c529c29cf53 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:29.686257 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a592f-870b-4252-92c6-5c529c29cf53 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:29.686734 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a592f-870b-4252-92c6-5c529c29cf53 None None] PATH_INFO: `/v3` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:29.688861 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 766/1517] 10.222.0.50 () {54 vars in 925 bytes} [Wed Aug 24 08:53:29 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:53:29.695585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d1d6b-df67-4b69-a2d8-eae17c895626 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:29.695894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d1d6b-df67-4b69-a2d8-eae17c895626 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:29.696195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d1d6b-df67-4b69-a2d8-eae17c895626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:29.804765 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9c2d1d6b-df67-4b69-a2d8-eae17c895626 None None] MFA Rules not processed for user `33a0844a1c8f4ea4932e12d53e5ee9ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:29.806760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9c2d1d6b-df67-4b69-a2d8-eae17c895626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:29.836441 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:53:29.914744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 752/1518] 10.222.0.50 () {58 vars in 1015 bytes} [Wed Aug 24 08:53:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:53:33.853878 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a27a5-61ab-4a93-a288-383a8916cc89 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:33.854428 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a27a5-61ab-4a93-a288-383a8916cc89 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:33.854583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a27a5-61ab-4a93-a288-383a8916cc89 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:33.855989 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 767/1519] 10.222.0.67 () {54 vars in 961 bytes} [Wed Aug 24 08:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:53:33.861875 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a462282-49c9-416c-8142-15d39d9407c2 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:33.862508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a462282-49c9-416c-8142-15d39d9407c2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:33.862508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a462282-49c9-416c-8142-15d39d9407c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:33.884534 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9a462282-49c9-416c-8142-15d39d9407c2 None None] MFA Rules not processed for user `33a0844a1c8f4ea4932e12d53e5ee9ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:33.885008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9a462282-49c9-416c-8142-15d39d9407c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:33.894637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 753/1520] 10.222.0.67 () {58 vars in 1058 bytes} [Wed Aug 24 08:53:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:53:33.903646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a27a5-61ab-4a93-a288-383a8916cc89 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:53:33.920833 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a27a5-61ab-4a93-a288-383a8916cc89 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:53:33.927102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed7f78-3b52-4bbf-9140-514554dacde1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33a0844a1c8f4ea4932e12d53e5ee9ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:53:33.927770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed7f78-3b52-4bbf-9140-514554dacde1 service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:33.928168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed7f78-3b52-4bbf-9140-514554dacde1 service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:33.928445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed7f78-3b52-4bbf-9140-514554dacde1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:33.937828 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed7f78-3b52-4bbf-9140-514554dacde1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:53:33.939457 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed7f78-3b52-4bbf-9140-514554dacde1 service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:53:33.951271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 768/1521] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:53:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:53:40.925017 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42390e-870f-45e5-bee8-4b20e49ab5eb None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:40.925695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42390e-870f-45e5-bee8-4b20e49ab5eb None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:40.925695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42390e-870f-45e5-bee8-4b20e49ab5eb None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:40.926936 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:53:40.934505 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4a98c-b2a5-4872-8fef-c94ecc023b6b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:40.935046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4a98c-b2a5-4872-8fef-c94ecc023b6b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:40.935460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4a98c-b2a5-4872-8fef-c94ecc023b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:40.961438 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-53a4a98c-b2a5-4872-8fef-c94ecc023b6b None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:40.962664 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-53a4a98c-b2a5-4872-8fef-c94ecc023b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:40.970977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:53:41.132029 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42390e-870f-45e5-bee8-4b20e49ab5eb None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:53:41.149131 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42390e-870f-45e5-bee8-4b20e49ab5eb None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:53:41.154761 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43ad7c-a39c-4257-b3da-8a13bf8e6d69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:53:41.155380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43ad7c-a39c-4257-b3da-8a13bf8e6d69 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:41.155659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43ad7c-a39c-4257-b3da-8a13bf8e6d69 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:41.155933 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43ad7c-a39c-4257-b3da-8a13bf8e6d69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:41.168032 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43ad7c-a39c-4257-b3da-8a13bf8e6d69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:53:41.169677 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43ad7c-a39c-4257-b3da-8a13bf8e6d69 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:53:41.181001 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 755/1524] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:53:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:53:42.083290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91dedde-3d7d-4623-81c5-0547d0bc0fd7 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:42.084135 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91dedde-3d7d-4623-81c5-0547d0bc0fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:42.084135 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91dedde-3d7d-4623-81c5-0547d0bc0fd7 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:42.085297 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 08:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:53:42.090685 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869a431-eb07-4433-973d-59cf32146181 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:42.091165 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869a431-eb07-4433-973d-59cf32146181 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:42.091269 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869a431-eb07-4433-973d-59cf32146181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:42.111750 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9869a431-eb07-4433-973d-59cf32146181 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:42.113057 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9869a431-eb07-4433-973d-59cf32146181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:42.123479 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 08:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:53:42.275595 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91dedde-3d7d-4623-81c5-0547d0bc0fd7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:53:42.290671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91dedde-3d7d-4623-81c5-0547d0bc0fd7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:53:42.296264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e09180-d1fb-4968-beb6-5b4cf820de0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:53:42.296807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e09180-d1fb-4968-beb6-5b4cf820de0c service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:42.297089 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e09180-d1fb-4968-beb6-5b4cf820de0c service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:42.297363 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e09180-d1fb-4968-beb6-5b4cf820de0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:42.306193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e09180-d1fb-4968-beb6-5b4cf820de0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:53:42.307804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e09180-d1fb-4968-beb6-5b4cf820de0c service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:53:42.318562 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 771/1527] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:53:46.836367 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a305b9a-1901-40ea-8289-91cf264e5001 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:46.836941 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a305b9a-1901-40ea-8289-91cf264e5001 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:46.836941 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a305b9a-1901-40ea-8289-91cf264e5001 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:46.838274 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:53:46.844429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41742902-48d5-4332-9e07-4ba48a407d03 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:46.844429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41742902-48d5-4332-9e07-4ba48a407d03 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:46.844968 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41742902-48d5-4332-9e07-4ba48a407d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:46.864209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-41742902-48d5-4332-9e07-4ba48a407d03 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:46.865989 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-41742902-48d5-4332-9e07-4ba48a407d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:46.874231 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:53:47.168607 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a305b9a-1901-40ea-8289-91cf264e5001 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:53:47.170487 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2a305b9a-1901-40ea-8289-91cf264e5001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:47.254742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a305b9a-1901-40ea-8289-91cf264e5001 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:53:47.258746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528791e-920f-43e8-9a9d-a1eaa5d4120d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:53:47.259453 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528791e-920f-43e8-9a9d-a1eaa5d4120d service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:47.259732 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528791e-920f-43e8-9a9d-a1eaa5d4120d service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:47.260004 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528791e-920f-43e8-9a9d-a1eaa5d4120d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:47.269977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528791e-920f-43e8-9a9d-a1eaa5d4120d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:53:47.271457 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528791e-920f-43e8-9a9d-a1eaa5d4120d service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:53:47.281101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 758/1530] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:53:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:53:49.321142 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a8b2d-03c3-4255-a0b8-c21e5019d9ff None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:49.321695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a8b2d-03c3-4255-a0b8-c21e5019d9ff None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:49.321695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a8b2d-03c3-4255-a0b8-c21e5019d9ff None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:49.323044 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:53:49.328162 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec8bec-0d91-470b-a6ca-85f125ef14fc None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:49.328596 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec8bec-0d91-470b-a6ca-85f125ef14fc None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:49.328728 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec8bec-0d91-470b-a6ca-85f125ef14fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:49.348987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9bec8bec-0d91-470b-a6ca-85f125ef14fc None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:49.351085 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9bec8bec-0d91-470b-a6ca-85f125ef14fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:49.358948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:53:49.649040 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a8b2d-03c3-4255-a0b8-c21e5019d9ff None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:53:49.660409 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a8b2d-03c3-4255-a0b8-c21e5019d9ff None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:53:49.664664 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c1197-2036-43f7-a089-a6365ade3ef9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:53:49.665352 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c1197-2036-43f7-a089-a6365ade3ef9 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:49.665632 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c1197-2036-43f7-a089-a6365ade3ef9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:49.665902 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c1197-2036-43f7-a089-a6365ade3ef9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:49.680162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38c1197-2036-43f7-a089-a6365ade3ef9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:53:49.681611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38c1197-2036-43f7-a089-a6365ade3ef9 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:53:49.691590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 774/1533] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:53:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:53:53.132575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327b0f2-4cd4-4032-8a32-eb49629a23af None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:53.133199 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327b0f2-4cd4-4032-8a32-eb49629a23af None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:53.133199 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327b0f2-4cd4-4032-8a32-eb49629a23af None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:53.134399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:53:53.139671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79781bee-0b15-4268-a61c-a71df7c24da2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:53.140180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79781bee-0b15-4268-a61c-a71df7c24da2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:53.140308 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79781bee-0b15-4268-a61c-a71df7c24da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:53.161441 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-79781bee-0b15-4268-a61c-a71df7c24da2 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:53.163416 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-79781bee-0b15-4268-a61c-a71df7c24da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:53.171783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:53:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:53:53.742221 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327b0f2-4cd4-4032-8a32-eb49629a23af None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:53:53.743967 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f327b0f2-4cd4-4032-8a32-eb49629a23af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:53.817583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:53:53.838826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327b0f2-4cd4-4032-8a32-eb49629a23af None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:53:53.842449 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28405e6-f441-4494-b136-66b1b77b8305 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:53:53.843056 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28405e6-f441-4494-b136-66b1b77b8305 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:53.843382 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28405e6-f441-4494-b136-66b1b77b8305 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:53.843706 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28405e6-f441-4494-b136-66b1b77b8305 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:53.855133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28405e6-f441-4494-b136-66b1b77b8305 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:53:53.856834 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28405e6-f441-4494-b136-66b1b77b8305 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:53:53.867529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 761/1536] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:53:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:53:55.844803 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec726f9d-6260-4c5c-a003-641cb7ab27db None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:55.845514 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec726f9d-6260-4c5c-a003-641cb7ab27db None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:55.845693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec726f9d-6260-4c5c-a003-641cb7ab27db None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:55.847063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 08:53:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:53:55.853909 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662d8e5-58d2-4354-a642-0f648fc2e25e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:55.854809 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662d8e5-58d2-4354-a642-0f648fc2e25e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:55.854975 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662d8e5-58d2-4354-a642-0f648fc2e25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:55.879237 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-6662d8e5-58d2-4354-a642-0f648fc2e25e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:53:55.880486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6662d8e5-58d2-4354-a642-0f648fc2e25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:55.889143 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 08:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:53:56.447702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec726f9d-6260-4c5c-a003-641cb7ab27db None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:53:56.449410 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ec726f9d-6260-4c5c-a003-641cb7ab27db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:53:56.456927 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=84 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:53:56.463825 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec726f9d-6260-4c5c-a003-641cb7ab27db None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:53:56.467634 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716aa07-2f7a-4102-889f-427f5abe2c30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:53:56.468344 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716aa07-2f7a-4102-889f-427f5abe2c30 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:53:56.468627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716aa07-2f7a-4102-889f-427f5abe2c30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:53:56.468902 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716aa07-2f7a-4102-889f-427f5abe2c30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:53:56.479958 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716aa07-2f7a-4102-889f-427f5abe2c30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:53:56.481432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716aa07-2f7a-4102-889f-427f5abe2c30 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:53:56.491850 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 777/1539] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:53:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:54:05.874471 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=11 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:54:06.486807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:24.399027 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2b931-bb3d-4f67-a9f9-a23289e76d61 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.399750 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2b931-bb3d-4f67-a9f9-a23289e76d61 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.400081 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2b931-bb3d-4f67-a9f9-a23289e76d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.401711 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d18a3-f91b-4265-9891-0e8abcb3d2d2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.402216 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d18a3-f91b-4265-9891-0e8abcb3d2d2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.402660 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d18a3-f91b-4265-9891-0e8abcb3d2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.431908 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-08e2b931-bb3d-4f67-a9f9-a23289e76d61 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:24.433185 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-08e2b931-bb3d-4f67-a9f9-a23289e76d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:24.453644 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 763/1540] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:24.462514 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0d8d18a3-f91b-4265-9891-0e8abcb3d2d2 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:24.463291 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdcfb4-8677-43a2-a97a-4b05c8814f4d None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.463574 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdcfb4-8677-43a2-a97a-4b05c8814f4d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.463845 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdcfb4-8677-43a2-a97a-4b05c8814f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.464611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0d8d18a3-f91b-4265-9891-0e8abcb3d2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:24.495417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 778/1541] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:24.506312 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca0fb9-ff26-4782-93f5-d04a32726c1c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.506784 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca0fb9-ff26-4782-93f5-d04a32726c1c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.507261 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca0fb9-ff26-4782-93f5-d04a32726c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.507670 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-4ecdcfb4-8677-43a2-a97a-4b05c8814f4d None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:24.509114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4ecdcfb4-8677-43a2-a97a-4b05c8814f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:24.525823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 764/1542] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:55:24.531638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdcfb4-8677-43a2-a97a-4b05c8814f4d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.550172 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdcfb4-8677-43a2-a97a-4b05c8814f4d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:24.563800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c811c7-d0f8-4ebe-8107-20bfce17cc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:24.565507 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c811c7-d0f8-4ebe-8107-20bfce17cc1e admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.565666 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-34ca0fb9-ff26-4782-93f5-d04a32726c1c None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:24.565773 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c811c7-d0f8-4ebe-8107-20bfce17cc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.565773 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c811c7-d0f8-4ebe-8107-20bfce17cc1e admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.565773 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c811c7-d0f8-4ebe-8107-20bfce17cc1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:24.566003 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c811c7-d0f8-4ebe-8107-20bfce17cc1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:24.567845 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c811c7-d0f8-4ebe-8107-20bfce17cc1e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:24.573550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-34ca0fb9-ff26-4782-93f5-d04a32726c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:24.600180 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 765/1543] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:24.609865 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c811c7-d0f8-4ebe-8107-20bfce17cc1e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.612626 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 779/1544] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:55:24.625796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca0fb9-ff26-4782-93f5-d04a32726c1c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.634491 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c811c7-d0f8-4ebe-8107-20bfce17cc1e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:24.650816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca0fb9-ff26-4782-93f5-d04a32726c1c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:24.652152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75dde69-e9f9-46ae-bed1-8bb276a1f746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:24.653148 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75dde69-e9f9-46ae-bed1-8bb276a1f746 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.653709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75dde69-e9f9-46ae-bed1-8bb276a1f746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.654245 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75dde69-e9f9-46ae-bed1-8bb276a1f746 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.655764 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75dde69-e9f9-46ae-bed1-8bb276a1f746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:24.657173 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: WARNING py.warnings [None req-f75dde69-e9f9-46ae-bed1-8bb276a1f746 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:55:24.657173 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: warnings.warn(msg) Aug 24 08:55:24.657173 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: Aug 24 08:55:24.659071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75dde69-e9f9-46ae-bed1-8bb276a1f746 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:24.670587 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37022c81-c521-4bc3-96bb-0e2ab87e3ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:24.670587 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37022c81-c521-4bc3-96bb-0e2ab87e3ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.670587 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37022c81-c521-4bc3-96bb-0e2ab87e3ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.670587 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37022c81-c521-4bc3-96bb-0e2ab87e3ff9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.671282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37022c81-c521-4bc3-96bb-0e2ab87e3ff9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:24.671682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37022c81-c521-4bc3-96bb-0e2ab87e3ff9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:24.673384 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-37022c81-c521-4bc3-96bb-0e2ab87e3ff9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:55:24.673384 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Aug 24 08:55:24.673384 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: Aug 24 08:55:24.674846 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37022c81-c521-4bc3-96bb-0e2ab87e3ff9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:24.718884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 780/1545] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:55:24.736317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37022c81-c521-4bc3-96bb-0e2ab87e3ff9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.764537 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37022c81-c521-4bc3-96bb-0e2ab87e3ff9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:24.767698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 766/1546] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/projects => generated 371 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:55:24.771897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75dde69-e9f9-46ae-bed1-8bb276a1f746 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.782737 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08917741-927f-4048-9ad7-3daee002a247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:24.783813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08917741-927f-4048-9ad7-3daee002a247 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:24.784328 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08917741-927f-4048-9ad7-3daee002a247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:24.784808 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08917741-927f-4048-9ad7-3daee002a247 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:24.785969 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08917741-927f-4048-9ad7-3daee002a247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:24.789790 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08917741-927f-4048-9ad7-3daee002a247 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:24.880804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 781/1547] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/projects => generated 375 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:24.889782 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08917741-927f-4048-9ad7-3daee002a247 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:24.903028 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75dde69-e9f9-46ae-bed1-8bb276a1f746 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.063310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcf0e1-7f5e-4546-abb5-3ef07bbc4877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.064431 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcf0e1-7f5e-4546-abb5-3ef07bbc4877 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.064933 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcf0e1-7f5e-4546-abb5-3ef07bbc4877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.065424 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcf0e1-7f5e-4546-abb5-3ef07bbc4877 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.066716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcf0e1-7f5e-4546-abb5-3ef07bbc4877 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.070918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcf0e1-7f5e-4546-abb5-3ef07bbc4877 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.078814 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08917741-927f-4048-9ad7-3daee002a247 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.092663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9610a-874f-45e2-af45-3b1f16ecedc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.093484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9610a-874f-45e2-af45-3b1f16ecedc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.093761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9610a-874f-45e2-af45-3b1f16ecedc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.094027 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9610a-874f-45e2-af45-3b1f16ecedc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.094681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9610a-874f-45e2-af45-3b1f16ecedc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.096255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-f9e9610a-874f-45e2-af45-3b1f16ecedc7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:55:25.096255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Aug 24 08:55:25.096255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: Aug 24 08:55:25.097696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9610a-874f-45e2-af45-3b1f16ecedc7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.112904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 767/1548] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/users => generated 286 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:25.115194 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c9d76-520d-4fce-b5df-2a16fd080cd7 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.115658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c9d76-520d-4fce-b5df-2a16fd080cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.116160 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c9d76-520d-4fce-b5df-2a16fd080cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.125161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 782/1549] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:24 2022] POST /identity/v3/users => generated 288 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:55:25.127187 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15940372-1929-48a4-bee6-e7b5ebccd276 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.127507 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15940372-1929-48a4-bee6-e7b5ebccd276 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.127843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15940372-1929-48a4-bee6-e7b5ebccd276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.145217 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-420c9d76-520d-4fce-b5df-2a16fd080cd7 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:25.146458 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-420c9d76-520d-4fce-b5df-2a16fd080cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:25.149367 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-15940372-1929-48a4-bee6-e7b5ebccd276 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:25.150552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-15940372-1929-48a4-bee6-e7b5ebccd276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:25.156498 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 768/1550] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:25.157796 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474194f8-fb3f-431b-9c72-5b55349a30a3 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.158076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474194f8-fb3f-431b-9c72-5b55349a30a3 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.158348 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474194f8-fb3f-431b-9c72-5b55349a30a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.167606 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 783/1551] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:25.167606 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd99dd-fddd-44e8-914c-6bc7dffa2e81 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.167606 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd99dd-fddd-44e8-914c-6bc7dffa2e81 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.167606 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd99dd-fddd-44e8-914c-6bc7dffa2e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.188924 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-474194f8-fb3f-431b-9c72-5b55349a30a3 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:25.189328 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-36bd99dd-fddd-44e8-914c-6bc7dffa2e81 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:25.198682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-474194f8-fb3f-431b-9c72-5b55349a30a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:25.199028 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-36bd99dd-fddd-44e8-914c-6bc7dffa2e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:25.199387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 769/1552] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:25.201281 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 784/1553] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:25.202498 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac836ab7-a8c6-4dbb-8359-4668da6c64f3 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.203166 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac836ab7-a8c6-4dbb-8359-4668da6c64f3 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.203576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3808634-bf2a-4b8c-9fe6-82eb113f910d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.203868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3808634-bf2a-4b8c-9fe6-82eb113f910d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.204168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3808634-bf2a-4b8c-9fe6-82eb113f910d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.204625 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac836ab7-a8c6-4dbb-8359-4668da6c64f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.232577 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c3808634-bf2a-4b8c-9fe6-82eb113f910d None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:25.234673 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c3808634-bf2a-4b8c-9fe6-82eb113f910d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:25.237577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-ac836ab7-a8c6-4dbb-8359-4668da6c64f3 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:25.239266 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ac836ab7-a8c6-4dbb-8359-4668da6c64f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:25.243585 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 785/1554] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:25.248797 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3808634-bf2a-4b8c-9fe6-82eb113f910d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.250734 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 770/1555] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:25.252108 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac836ab7-a8c6-4dbb-8359-4668da6c64f3 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.258810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3808634-bf2a-4b8c-9fe6-82eb113f910d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.270671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac836ab7-a8c6-4dbb-8359-4668da6c64f3 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.271134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5c6f0-5098-41a9-8232-835076390d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.271769 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5c6f0-5098-41a9-8232-835076390d74 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.272063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5c6f0-5098-41a9-8232-835076390d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.272378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5c6f0-5098-41a9-8232-835076390d74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.273039 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5c6f0-5098-41a9-8232-835076390d74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.274692 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5c6f0-5098-41a9-8232-835076390d74 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.277491 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f2731-f14f-46b0-acd6-3777b55dbf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.278340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f2731-f14f-46b0-acd6-3777b55dbf29 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.278803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f2731-f14f-46b0-acd6-3777b55dbf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.279120 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f2731-f14f-46b0-acd6-3777b55dbf29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.280370 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f2731-f14f-46b0-acd6-3777b55dbf29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.283202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f2731-f14f-46b0-acd6-3777b55dbf29 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.286536 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 786/1556] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:25.288325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b36b1-b52c-4a0d-88b7-6e8913e43411 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.288582 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b36b1-b52c-4a0d-88b7-6e8913e43411 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.288849 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b36b1-b52c-4a0d-88b7-6e8913e43411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.297144 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 771/1557] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:25.302808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4942e-0b65-499e-bf9b-d465140bfab2 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.302808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4942e-0b65-499e-bf9b-d465140bfab2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.302808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4942e-0b65-499e-bf9b-d465140bfab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.318010 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-341b36b1-b52c-4a0d-88b7-6e8913e43411 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:25.319394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-341b36b1-b52c-4a0d-88b7-6e8913e43411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:25.319894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-20b4942e-0b65-499e-bf9b-d465140bfab2 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:25.321075 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-20b4942e-0b65-499e-bf9b-d465140bfab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:25.328482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 787/1558] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:25.330257 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 772/1559] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:25.331525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76066ab3-42c7-4296-b299-efa82799c1d1 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.331808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76066ab3-42c7-4296-b299-efa82799c1d1 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.332155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76066ab3-42c7-4296-b299-efa82799c1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.332503 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54222797-2fb9-4818-bb24-8b958a939a74 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.332980 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54222797-2fb9-4818-bb24-8b958a939a74 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.333441 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54222797-2fb9-4818-bb24-8b958a939a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.339715 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:25.359973 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-54222797-2fb9-4818-bb24-8b958a939a74 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:25.361241 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-54222797-2fb9-4818-bb24-8b958a939a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:25.362344 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-76066ab3-42c7-4296-b299-efa82799c1d1 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:25.363555 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-76066ab3-42c7-4296-b299-efa82799c1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:25.369770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 788/1560] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:25.371116 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a430c7-a2af-42b1-9492-12508f463ff6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.371399 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a430c7-a2af-42b1-9492-12508f463ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.371664 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a430c7-a2af-42b1-9492-12508f463ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.377956 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 773/1561] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:25.380006 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0886591-7f86-4c53-bd30-f2879e8f8e69 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.380526 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0886591-7f86-4c53-bd30-f2879e8f8e69 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.380889 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0886591-7f86-4c53-bd30-f2879e8f8e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.398748 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-35a430c7-a2af-42b1-9492-12508f463ff6 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:25.400081 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-35a430c7-a2af-42b1-9492-12508f463ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:25.404395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b0886591-7f86-4c53-bd30-f2879e8f8e69 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:25.406823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b0886591-7f86-4c53-bd30-f2879e8f8e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:25.414907 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 789/1562] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:25.421148 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 774/1563] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:25.421148 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0886591-7f86-4c53-bd30-f2879e8f8e69 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.421535 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a430c7-a2af-42b1-9492-12508f463ff6 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.431627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a430c7-a2af-42b1-9492-12508f463ff6 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.431920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0886591-7f86-4c53-bd30-f2879e8f8e69 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.439013 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0551f-cdfa-4a15-9fb1-82d2ce039455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.439384 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370175c-bec2-45c9-b135-7301fd1f02a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.440452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370175c-bec2-45c9-b135-7301fd1f02a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.440938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370175c-bec2-45c9-b135-7301fd1f02a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.441147 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0551f-cdfa-4a15-9fb1-82d2ce039455 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.441408 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370175c-bec2-45c9-b135-7301fd1f02a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.441771 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0551f-cdfa-4a15-9fb1-82d2ce039455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.442220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0551f-cdfa-4a15-9fb1-82d2ce039455 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.442537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a370175c-bec2-45c9-b135-7301fd1f02a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.443131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb0551f-cdfa-4a15-9fb1-82d2ce039455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.444912 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb0551f-cdfa-4a15-9fb1-82d2ce039455 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.445594 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a370175c-bec2-45c9-b135-7301fd1f02a9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.461069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 790/1564] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:25.461828 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0551f-cdfa-4a15-9fb1-82d2ce039455 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.470589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 775/1565] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:25.470589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370175c-bec2-45c9-b135-7301fd1f02a9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.477910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0551f-cdfa-4a15-9fb1-82d2ce039455 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.485883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9232e1f-34e5-4fce-8825-1eb56f6d9c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.486859 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9232e1f-34e5-4fce-8825-1eb56f6d9c5d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.487319 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9232e1f-34e5-4fce-8825-1eb56f6d9c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.487775 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9232e1f-34e5-4fce-8825-1eb56f6d9c5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.488311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370175c-bec2-45c9-b135-7301fd1f02a9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.488874 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9232e1f-34e5-4fce-8825-1eb56f6d9c5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:25.489392 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9232e1f-34e5-4fce-8825-1eb56f6d9c5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.492112 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9232e1f-34e5-4fce-8825-1eb56f6d9c5d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.494517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb048ae-7a3e-4a43-aae0-4abf34fd8c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.495124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb048ae-7a3e-4a43-aae0-4abf34fd8c04 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.495408 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb048ae-7a3e-4a43-aae0-4abf34fd8c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.495678 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb048ae-7a3e-4a43-aae0-4abf34fd8c04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.496367 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb048ae-7a3e-4a43-aae0-4abf34fd8c04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:25.496668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb048ae-7a3e-4a43-aae0-4abf34fd8c04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.498918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb048ae-7a3e-4a43-aae0-4abf34fd8c04 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.506886 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 791/1566] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:25.507565 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9232e1f-34e5-4fce-8825-1eb56f6d9c5d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.513072 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 776/1567] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:25.514140 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb048ae-7a3e-4a43-aae0-4abf34fd8c04 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.520210 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9232e1f-34e5-4fce-8825-1eb56f6d9c5d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.524623 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb048ae-7a3e-4a43-aae0-4abf34fd8c04 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.525855 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aef363-3ed5-466d-8d67-87c1da70bd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.526478 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aef363-3ed5-466d-8d67-87c1da70bd7f admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.526757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aef363-3ed5-466d-8d67-87c1da70bd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.527022 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aef363-3ed5-466d-8d67-87c1da70bd7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.528192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aef363-3ed5-466d-8d67-87c1da70bd7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:25.528192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aef363-3ed5-466d-8d67-87c1da70bd7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.529661 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aef363-3ed5-466d-8d67-87c1da70bd7f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.529809 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38489e2e-ce6d-42f2-b78b-9e5b5aef0cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.530751 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38489e2e-ce6d-42f2-b78b-9e5b5aef0cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.531039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38489e2e-ce6d-42f2-b78b-9e5b5aef0cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.531414 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38489e2e-ce6d-42f2-b78b-9e5b5aef0cf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.531963 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38489e2e-ce6d-42f2-b78b-9e5b5aef0cf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:25.532295 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38489e2e-ce6d-42f2-b78b-9e5b5aef0cf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.533882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38489e2e-ce6d-42f2-b78b-9e5b5aef0cf1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.544119 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 792/1568] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:25.544986 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aef363-3ed5-466d-8d67-87c1da70bd7f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.546835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 777/1569] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:25.547535 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38489e2e-ce6d-42f2-b78b-9e5b5aef0cf1 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.562573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aef363-3ed5-466d-8d67-87c1da70bd7f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.562847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38489e2e-ce6d-42f2-b78b-9e5b5aef0cf1 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.567954 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156cf7d4-76bb-4b38-8f54-e41d4a5a211d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.568282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b382b9-9019-4172-a7e6-b95c96a48701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.568609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156cf7d4-76bb-4b38-8f54-e41d4a5a211d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.568889 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156cf7d4-76bb-4b38-8f54-e41d4a5a211d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.569156 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156cf7d4-76bb-4b38-8f54-e41d4a5a211d admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.569263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b382b9-9019-4172-a7e6-b95c96a48701 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.569613 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b382b9-9019-4172-a7e6-b95c96a48701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.569803 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156cf7d4-76bb-4b38-8f54-e41d4a5a211d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:25.570077 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b382b9-9019-4172-a7e6-b95c96a48701 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.570184 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156cf7d4-76bb-4b38-8f54-e41d4a5a211d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.570902 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b382b9-9019-4172-a7e6-b95c96a48701 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:25.571304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b382b9-9019-4172-a7e6-b95c96a48701 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.571783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156cf7d4-76bb-4b38-8f54-e41d4a5a211d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.573121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b382b9-9019-4172-a7e6-b95c96a48701 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.581695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:25.581695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 793/1570] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:25.588145 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 778/1571] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:25.589835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b382b9-9019-4172-a7e6-b95c96a48701 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.591016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156cf7d4-76bb-4b38-8f54-e41d4a5a211d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.606423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156cf7d4-76bb-4b38-8f54-e41d4a5a211d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.607406 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b382b9-9019-4172-a7e6-b95c96a48701 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.613158 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f87869-3257-4427-92b0-e62f4bc22b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.613837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f87869-3257-4427-92b0-e62f4bc22b49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.614157 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f87869-3257-4427-92b0-e62f4bc22b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.614397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f87869-3257-4427-92b0-e62f4bc22b49 admin admin] PATH_INFO: `/v3/projects/bd6e1733fd9d4f5096c35e99dd56945b/users/f0dc2dcc5327489c9ecc28a43ab7c0e2/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.615098 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266e03e-e06b-4e9a-bc14-054b98f3f217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.616319 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266e03e-e06b-4e9a-bc14-054b98f3f217 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.616852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266e03e-e06b-4e9a-bc14-054b98f3f217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.617336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266e03e-e06b-4e9a-bc14-054b98f3f217 admin admin] PATH_INFO: `/v3/projects/dba44ee9cc0c427a81b9c539b056e30d/users/e84cbb1c1968413989870678d1165d17/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.631751 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f87869-3257-4427-92b0-e62f4bc22b49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd6e1733fd9d4f5096c35e99dd56945b, user_id=f0dc2dcc5327489c9ecc28a43ab7c0e2, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.632853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: WARNING py.warnings [None req-d6f87869-3257-4427-92b0-e62f4bc22b49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:55:25.632853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: warnings.warn(msg) Aug 24 08:55:25.632853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: Aug 24 08:55:25.634856 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f87869-3257-4427-92b0-e62f4bc22b49 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.642239 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266e03e-e06b-4e9a-bc14-054b98f3f217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba44ee9cc0c427a81b9c539b056e30d, user_id=e84cbb1c1968413989870678d1165d17, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.646128 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266e03e-e06b-4e9a-bc14-054b98f3f217 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.650668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 779/1572] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:25 2022] PUT /identity/v3/projects/bd6e1733fd9d4f5096c35e99dd56945b/users/f0dc2dcc5327489c9ecc28a43ab7c0e2/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:25.651511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f87869-3257-4427-92b0-e62f4bc22b49 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.671753 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 794/1573] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:25 2022] PUT /identity/v3/projects/dba44ee9cc0c427a81b9c539b056e30d/users/e84cbb1c1968413989870678d1165d17/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:25.672826 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266e03e-e06b-4e9a-bc14-054b98f3f217 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.748578 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f87869-3257-4427-92b0-e62f4bc22b49 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.753805 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1f30a-ea3a-4f09-b30a-d477bbc11297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.754421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266e03e-e06b-4e9a-bc14-054b98f3f217 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.754560 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1f30a-ea3a-4f09-b30a-d477bbc11297 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.754696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1f30a-ea3a-4f09-b30a-d477bbc11297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.754965 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1f30a-ea3a-4f09-b30a-d477bbc11297 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.755608 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1f30a-ea3a-4f09-b30a-d477bbc11297 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.757561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1f30a-ea3a-4f09-b30a-d477bbc11297 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.759389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ed6e0-71e8-46df-816c-f95f282e9704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.760014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ed6e0-71e8-46df-816c-f95f282e9704 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.760329 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ed6e0-71e8-46df-816c-f95f282e9704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.760603 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ed6e0-71e8-46df-816c-f95f282e9704 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.761237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ed6e0-71e8-46df-816c-f95f282e9704 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.763174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ed6e0-71e8-46df-816c-f95f282e9704 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.790050 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 780/1574] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/projects => generated 389 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:25.790874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1f30a-ea3a-4f09-b30a-d477bbc11297 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.799944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 795/1575] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:25.799944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ed6e0-71e8-46df-816c-f95f282e9704 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.864263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1f30a-ea3a-4f09-b30a-d477bbc11297 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.864967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ed6e0-71e8-46df-816c-f95f282e9704 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.868989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc372100-9258-495c-a40a-b4291886eece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.869764 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc372100-9258-495c-a40a-b4291886eece admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.870174 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc372100-9258-495c-a40a-b4291886eece admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.870303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc372100-9258-495c-a40a-b4291886eece admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.870943 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc372100-9258-495c-a40a-b4291886eece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.872925 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8380d6-db7c-447d-b630-feeaf73afb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.873079 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc372100-9258-495c-a40a-b4291886eece admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.873552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8380d6-db7c-447d-b630-feeaf73afb39 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.873833 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8380d6-db7c-447d-b630-feeaf73afb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:25.874104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8380d6-db7c-447d-b630-feeaf73afb39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:25.875014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8380d6-db7c-447d-b630-feeaf73afb39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:25.877007 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8380d6-db7c-447d-b630-feeaf73afb39 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:25.878404 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:25.906724 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 781/1576] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:25.909031 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 796/1577] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/projects => generated 391 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:25.910874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc372100-9258-495c-a40a-b4291886eece admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.911026 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8380d6-db7c-447d-b630-feeaf73afb39 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:25.977000 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:25.993469 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc372100-9258-495c-a40a-b4291886eece admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.994463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8380d6-db7c-447d-b630-feeaf73afb39 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:25.998815 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438afcdb-5827-43df-8fce-bd3248e9e6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.999552 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438afcdb-5827-43df-8fce-bd3248e9e6df admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:25.999804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb8183-00eb-42dc-99b6-7ac4dc44b238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:25.999953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438afcdb-5827-43df-8fce-bd3248e9e6df admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.000154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438afcdb-5827-43df-8fce-bd3248e9e6df admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.000452 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb8183-00eb-42dc-99b6-7ac4dc44b238 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.000732 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb8183-00eb-42dc-99b6-7ac4dc44b238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.000858 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438afcdb-5827-43df-8fce-bd3248e9e6df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.001002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb8183-00eb-42dc-99b6-7ac4dc44b238 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.001641 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb8183-00eb-42dc-99b6-7ac4dc44b238 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.002699 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438afcdb-5827-43df-8fce-bd3248e9e6df admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.003541 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb8183-00eb-42dc-99b6-7ac4dc44b238 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.036278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 797/1578] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:26.036278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d59f0-69f8-4faf-b7e7-7c334c981409 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.036278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d59f0-69f8-4faf-b7e7-7c334c981409 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.036278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d59f0-69f8-4faf-b7e7-7c334c981409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.036946 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 782/1579] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:25 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:26.038370 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d94bdf-7393-4f0f-ad84-5bc1cdf80090 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.038660 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d94bdf-7393-4f0f-ad84-5bc1cdf80090 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.039090 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d94bdf-7393-4f0f-ad84-5bc1cdf80090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.055342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-359d59f0-69f8-4faf-b7e7-7c334c981409 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:26.056625 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-359d59f0-69f8-4faf-b7e7-7c334c981409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:26.070243 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-c3d94bdf-7393-4f0f-ad84-5bc1cdf80090 None None] MFA Rules not processed for user `e84cbb1c1968413989870678d1165d17`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:26.071442 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c3d94bdf-7393-4f0f-ad84-5bc1cdf80090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:26.135543 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 798/1580] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:26.136290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d59f0-69f8-4faf-b7e7-7c334c981409 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.146808 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d59f0-69f8-4faf-b7e7-7c334c981409 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.153668 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596492c-5015-48c9-8746-c1622ef4cc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.154342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596492c-5015-48c9-8746-c1622ef4cc8b admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.154679 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596492c-5015-48c9-8746-c1622ef4cc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.155200 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596492c-5015-48c9-8746-c1622ef4cc8b admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.155857 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596492c-5015-48c9-8746-c1622ef4cc8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.157055 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 783/1581] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:26.161531 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596492c-5015-48c9-8746-c1622ef4cc8b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.161701 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d94bdf-7393-4f0f-ad84-5bc1cdf80090 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.180594 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d94bdf-7393-4f0f-ad84-5bc1cdf80090 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.187165 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cca48d-e08f-4c4d-9677-f7cb57643e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.187841 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cca48d-e08f-4c4d-9677-f7cb57643e6b admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.188229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cca48d-e08f-4c4d-9677-f7cb57643e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.188844 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cca48d-e08f-4c4d-9677-f7cb57643e6b admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.189957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cca48d-e08f-4c4d-9677-f7cb57643e6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.195089 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 799/1582] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:26 2022] POST /identity/v3/users => generated 295 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:26.195292 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cca48d-e08f-4c4d-9677-f7cb57643e6b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.195419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596492c-5015-48c9-8746-c1622ef4cc8b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.210448 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596492c-5015-48c9-8746-c1622ef4cc8b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.217201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b5ff8-bef4-41b0-a38c-2793f698571d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.218008 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b5ff8-bef4-41b0-a38c-2793f698571d admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.218822 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b5ff8-bef4-41b0-a38c-2793f698571d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.220847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b5ff8-bef4-41b0-a38c-2793f698571d admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.222291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b5ff8-bef4-41b0-a38c-2793f698571d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.289187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 784/1583] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:26 2022] POST /identity/v3/users => generated 289 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:26.289187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cca48d-e08f-4c4d-9677-f7cb57643e6b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.289610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b5ff8-bef4-41b0-a38c-2793f698571d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.289738 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cca48d-e08f-4c4d-9677-f7cb57643e6b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.289738 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640e172-c8dd-455b-9604-57021727aa18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.289738 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640e172-c8dd-455b-9604-57021727aa18 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.289738 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640e172-c8dd-455b-9604-57021727aa18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.290056 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640e172-c8dd-455b-9604-57021727aa18 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.290143 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 800/1584] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:26 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:26.290223 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640e172-c8dd-455b-9604-57021727aa18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.290313 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b5ff8-bef4-41b0-a38c-2793f698571d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.290389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640e172-c8dd-455b-9604-57021727aa18 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.300775 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b5ff8-bef4-41b0-a38c-2793f698571d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.307357 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b6ae9-1fbd-4dd2-8c3f-2f22cec525ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.308605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b6ae9-1fbd-4dd2-8c3f-2f22cec525ee admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.309008 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b6ae9-1fbd-4dd2-8c3f-2f22cec525ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.309488 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b6ae9-1fbd-4dd2-8c3f-2f22cec525ee admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.315090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b6ae9-1fbd-4dd2-8c3f-2f22cec525ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.317521 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b6ae9-1fbd-4dd2-8c3f-2f22cec525ee admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.335050 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 785/1585] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:26 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:26.336865 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640e172-c8dd-455b-9604-57021727aa18 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.351456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 801/1586] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:26 2022] POST /identity/v3/users => generated 292 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:26.352979 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b6ae9-1fbd-4dd2-8c3f-2f22cec525ee admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.373146 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640e172-c8dd-455b-9604-57021727aa18 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.398250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b8792-40cb-4e17-99c0-29e768caf1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.398779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b8792-40cb-4e17-99c0-29e768caf1ce admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.399064 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b8792-40cb-4e17-99c0-29e768caf1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.399331 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b8792-40cb-4e17-99c0-29e768caf1ce admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.399972 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b8792-40cb-4e17-99c0-29e768caf1ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.403072 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b8792-40cb-4e17-99c0-29e768caf1ce admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.431239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 786/1587] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:26 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:26.436366 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e766e-5d7c-494f-98f1-9d00f2bcb7bc None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.436918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e766e-5d7c-494f-98f1-9d00f2bcb7bc None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.437438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e766e-5d7c-494f-98f1-9d00f2bcb7bc None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.438868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b6ae9-1fbd-4dd2-8c3f-2f22cec525ee admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.439493 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 787/1588] 10.222.0.67 () {54 vars in 962 bytes} [Wed Aug 24 08:55:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:55:26.440120 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e766e-5d7c-494f-98f1-9d00f2bcb7bc None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.450046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570ac78-1005-424f-8d7f-71a2ffe1bd16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.450742 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570ac78-1005-424f-8d7f-71a2ffe1bd16 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.451496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570ac78-1005-424f-8d7f-71a2ffe1bd16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.452296 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570ac78-1005-424f-8d7f-71a2ffe1bd16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.458795 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e766e-5d7c-494f-98f1-9d00f2bcb7bc None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.465281 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570ac78-1005-424f-8d7f-71a2ffe1bd16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.465606 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef475ad-fea3-41e3-bbae-675336862f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.466494 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef475ad-fea3-41e3-bbae-675336862f93 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.467566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570ac78-1005-424f-8d7f-71a2ffe1bd16 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.467903 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef475ad-fea3-41e3-bbae-675336862f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.468400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef475ad-fea3-41e3-bbae-675336862f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.469956 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef475ad-fea3-41e3-bbae-675336862f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.472298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef475ad-fea3-41e3-bbae-675336862f93 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.481596 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:26.485265 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 802/1589] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:26.486362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570ac78-1005-424f-8d7f-71a2ffe1bd16 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.489176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 788/1590] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:26.490630 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef475ad-fea3-41e3-bbae-675336862f93 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.508226 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570ac78-1005-424f-8d7f-71a2ffe1bd16 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.511071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef475ad-fea3-41e3-bbae-675336862f93 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.517455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73694f-bb7d-4d58-98f7-597f7423545d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.518453 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73694f-bb7d-4d58-98f7-597f7423545d admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.518902 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73694f-bb7d-4d58-98f7-597f7423545d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.519335 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73694f-bb7d-4d58-98f7-597f7423545d admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.520166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f95fbb-6619-49f2-99be-29bed8d18b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.520821 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f95fbb-6619-49f2-99be-29bed8d18b6d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.520957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73694f-bb7d-4d58-98f7-597f7423545d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.521127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f95fbb-6619-49f2-99be-29bed8d18b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.521373 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f95fbb-6619-49f2-99be-29bed8d18b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.522121 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f95fbb-6619-49f2-99be-29bed8d18b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.522810 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73694f-bb7d-4d58-98f7-597f7423545d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.523754 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f95fbb-6619-49f2-99be-29bed8d18b6d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.535631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 789/1591] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:26.539021 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73694f-bb7d-4d58-98f7-597f7423545d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.543401 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 803/1592] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:26.543401 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f95fbb-6619-49f2-99be-29bed8d18b6d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.566441 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73694f-bb7d-4d58-98f7-597f7423545d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.568838 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f95fbb-6619-49f2-99be-29bed8d18b6d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.573956 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6629524-8801-4485-80ca-0e682944564c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.575157 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6629524-8801-4485-80ca-0e682944564c admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.575587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6629524-8801-4485-80ca-0e682944564c admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.576245 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6629524-8801-4485-80ca-0e682944564c admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.577949 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6629524-8801-4485-80ca-0e682944564c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.579765 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6629524-8801-4485-80ca-0e682944564c admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.583398 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff29289-fbbf-4867-9528-b5682d276cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.584024 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff29289-fbbf-4867-9528-b5682d276cfc admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.584436 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff29289-fbbf-4867-9528-b5682d276cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.584602 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff29289-fbbf-4867-9528-b5682d276cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.585258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff29289-fbbf-4867-9528-b5682d276cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.586866 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff29289-fbbf-4867-9528-b5682d276cfc admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.595796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:26.598078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 790/1593] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:26.599146 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6629524-8801-4485-80ca-0e682944564c admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.614809 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 804/1594] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:26.617067 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a82c4-4cda-45de-9ae3-cbac8e12e0c0 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.617067 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a82c4-4cda-45de-9ae3-cbac8e12e0c0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.617067 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a82c4-4cda-45de-9ae3-cbac8e12e0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.618242 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6629524-8801-4485-80ca-0e682944564c admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.629215 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cf174-2a5f-4abf-bc1a-a07dad0edd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.630159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cf174-2a5f-4abf-bc1a-a07dad0edd0d admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.630801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cf174-2a5f-4abf-bc1a-a07dad0edd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.631285 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cf174-2a5f-4abf-bc1a-a07dad0edd0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.632187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0cf174-2a5f-4abf-bc1a-a07dad0edd0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.634054 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0cf174-2a5f-4abf-bc1a-a07dad0edd0d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.653232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 791/1595] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:26.653232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cf174-2a5f-4abf-bc1a-a07dad0edd0d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.674348 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cf174-2a5f-4abf-bc1a-a07dad0edd0d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.682543 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1790a5f-cc4a-4c76-a487-60d66e0c764d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.684757 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1790a5f-cc4a-4c76-a487-60d66e0c764d admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.685283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1790a5f-cc4a-4c76-a487-60d66e0c764d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.686815 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1790a5f-cc4a-4c76-a487-60d66e0c764d admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.686956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-081a82c4-4cda-45de-9ae3-cbac8e12e0c0 None None] MFA Rules not processed for user `ca56dd7216fb4946946b2801e144fec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:26.688129 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-081a82c4-4cda-45de-9ae3-cbac8e12e0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:26.689957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1790a5f-cc4a-4c76-a487-60d66e0c764d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.692988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1790a5f-cc4a-4c76-a487-60d66e0c764d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.709603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 792/1596] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:26.709603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1790a5f-cc4a-4c76-a487-60d66e0c764d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.719987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1790a5f-cc4a-4c76-a487-60d66e0c764d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.727047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b4533-3476-4b51-bd41-1837978babcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.728418 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b4533-3476-4b51-bd41-1837978babcd admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.728693 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b4533-3476-4b51-bd41-1837978babcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.728968 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b4533-3476-4b51-bd41-1837978babcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.729657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b4533-3476-4b51-bd41-1837978babcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.731650 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b4533-3476-4b51-bd41-1837978babcd admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.747759 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 793/1597] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:26.749121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b4533-3476-4b51-bd41-1837978babcd admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.760883 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b4533-3476-4b51-bd41-1837978babcd admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.766833 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bae233-444a-4d9f-b6ff-f7c8d2bb6864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.767600 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bae233-444a-4d9f-b6ff-f7c8d2bb6864 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.767983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bae233-444a-4d9f-b6ff-f7c8d2bb6864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.768391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bae233-444a-4d9f-b6ff-f7c8d2bb6864 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.769160 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bae233-444a-4d9f-b6ff-f7c8d2bb6864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.770860 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bae233-444a-4d9f-b6ff-f7c8d2bb6864 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.787671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 794/1598] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:26.787671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bae233-444a-4d9f-b6ff-f7c8d2bb6864 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.795616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bae233-444a-4d9f-b6ff-f7c8d2bb6864 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.800895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca64bc7-f7e8-4bee-98c3-33e39432d605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.801513 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca64bc7-f7e8-4bee-98c3-33e39432d605 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.801799 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca64bc7-f7e8-4bee-98c3-33e39432d605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.802072 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca64bc7-f7e8-4bee-98c3-33e39432d605 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.802729 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca64bc7-f7e8-4bee-98c3-33e39432d605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.804389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca64bc7-f7e8-4bee-98c3-33e39432d605 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.810050 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 805/1599] 10.222.0.67 () {58 vars in 1059 bytes} [Wed Aug 24 08:55:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:26.811003 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a82c4-4cda-45de-9ae3-cbac8e12e0c0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.821900 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a82c4-4cda-45de-9ae3-cbac8e12e0c0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.827820 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a79ec-8b3d-4a13-a1a4-231ca8cfb41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.828495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a79ec-8b3d-4a13-a1a4-231ca8cfb41a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.828777 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a79ec-8b3d-4a13-a1a4-231ca8cfb41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.829079 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a79ec-8b3d-4a13-a1a4-231ca8cfb41a admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.829748 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a79ec-8b3d-4a13-a1a4-231ca8cfb41a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.830220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 795/1600] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:26.831314 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca64bc7-f7e8-4bee-98c3-33e39432d605 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.831513 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a79ec-8b3d-4a13-a1a4-231ca8cfb41a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.844424 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca64bc7-f7e8-4bee-98c3-33e39432d605 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.844694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 806/1601] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:26.848685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a79ec-8b3d-4a13-a1a4-231ca8cfb41a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.851204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3591212-d791-48a8-93fe-b6dfb927719c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.852129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3591212-d791-48a8-93fe-b6dfb927719c admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.852838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3591212-d791-48a8-93fe-b6dfb927719c admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.853346 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3591212-d791-48a8-93fe-b6dfb927719c admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.854324 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3591212-d791-48a8-93fe-b6dfb927719c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.856670 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3591212-d791-48a8-93fe-b6dfb927719c admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.859733 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a79ec-8b3d-4a13-a1a4-231ca8cfb41a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.866250 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a4559-6cfb-4251-bf37-1df5727ad7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.867376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a4559-6cfb-4251-bf37-1df5727ad7f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.867376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a4559-6cfb-4251-bf37-1df5727ad7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.868456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a4559-6cfb-4251-bf37-1df5727ad7f5 admin admin] PATH_INFO: `/v3/projects/edeca01476604a70bed160e5aa97607c/users/f0d0eb9dc0f743c3985218901334b336/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.871293 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 796/1602] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:26.872144 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3591212-d791-48a8-93fe-b6dfb927719c admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.884235 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3591212-d791-48a8-93fe-b6dfb927719c admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:26.892562 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a4559-6cfb-4251-bf37-1df5727ad7f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edeca01476604a70bed160e5aa97607c, user_id=f0d0eb9dc0f743c3985218901334b336, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.892903 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e748841-80f9-4e17-b216-d0210a5a4aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:26.892903 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e748841-80f9-4e17-b216-d0210a5a4aa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:26.893213 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e748841-80f9-4e17-b216-d0210a5a4aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:26.893464 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e748841-80f9-4e17-b216-d0210a5a4aa4 admin admin] PATH_INFO: `/v3/projects/a586b9a96c934f9db30983d83142208f/users/db10227415804c2287ec4a60a6cbd2d5/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:26.894895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a4559-6cfb-4251-bf37-1df5727ad7f5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.911905 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e748841-80f9-4e17-b216-d0210a5a4aa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a586b9a96c934f9db30983d83142208f, user_id=db10227415804c2287ec4a60a6cbd2d5, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:26.916637 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 807/1603] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:26 2022] PUT /identity/v3/projects/edeca01476604a70bed160e5aa97607c/users/f0d0eb9dc0f743c3985218901334b336/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:26.916637 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a4559-6cfb-4251-bf37-1df5727ad7f5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:26.916897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e748841-80f9-4e17-b216-d0210a5a4aa4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:26.944812 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 797/1604] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:26 2022] PUT /identity/v3/projects/a586b9a96c934f9db30983d83142208f/users/db10227415804c2287ec4a60a6cbd2d5/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:26.944812 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e748841-80f9-4e17-b216-d0210a5a4aa4 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:27.046973 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a4559-6cfb-4251-bf37-1df5727ad7f5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:27.063497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e748841-80f9-4e17-b216-d0210a5a4aa4 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:27.071374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ee62a-4b96-4b33-8b58-933a7b2fc551 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:27.072744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ee62a-4b96-4b33-8b58-933a7b2fc551 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.073533 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ee62a-4b96-4b33-8b58-933a7b2fc551 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.073984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ee62a-4b96-4b33-8b58-933a7b2fc551 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.106821 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:27.124638 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace98a76-7297-45f0-8500-ebc88bf81f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:27.125529 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace98a76-7297-45f0-8500-ebc88bf81f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.125996 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace98a76-7297-45f0-8500-ebc88bf81f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.126307 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace98a76-7297-45f0-8500-ebc88bf81f0e admin admin] PATH_INFO: `/v3/projects/39195166aed84c998b13b926c0a39ab7/users/5c036cb4ba8e4a16bd9e884243e665c1/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.127952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:27.144730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace98a76-7297-45f0-8500-ebc88bf81f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=39195166aed84c998b13b926c0a39ab7, user_id=5c036cb4ba8e4a16bd9e884243e665c1, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:27.147686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace98a76-7297-45f0-8500-ebc88bf81f0e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:27.151334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ee62a-4b96-4b33-8b58-933a7b2fc551 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:27.153620 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ee62a-4b96-4b33-8b58-933a7b2fc551 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:27.165352 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 798/1605] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:27.169521 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 808/1606] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:26 2022] PUT /identity/v3/projects/39195166aed84c998b13b926c0a39ab7/users/5c036cb4ba8e4a16bd9e884243e665c1/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:27.173164 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ee62a-4b96-4b33-8b58-933a7b2fc551 service nova] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:27.174622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace98a76-7297-45f0-8500-ebc88bf81f0e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:27.194911 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ee62a-4b96-4b33-8b58-933a7b2fc551 service nova] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:27.271788 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2a68b-b3c2-4354-89ff-cc9bad48abcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:27.273723 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2a68b-b3c2-4354-89ff-cc9bad48abcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.274453 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2a68b-b3c2-4354-89ff-cc9bad48abcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.274907 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2a68b-b3c2-4354-89ff-cc9bad48abcc admin admin] PATH_INFO: `/v3/projects/b5cb111df3ec4249a836518a598063a4/users/8ee5364270094596b64b08a8ac56f6ce/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.276880 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace98a76-7297-45f0-8500-ebc88bf81f0e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:27.293239 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e998470-214e-4e6a-8734-1a319825e06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:27.294770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e998470-214e-4e6a-8734-1a319825e06c admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.296604 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e998470-214e-4e6a-8734-1a319825e06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.297285 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e998470-214e-4e6a-8734-1a319825e06c admin admin] PATH_INFO: `/v3/projects/7e6183eeb3ff43e79eccb5d7ad08c53c/users/bd0929f371e4442eb4eb6e12a96bbfc4/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.307859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2a68b-b3c2-4354-89ff-cc9bad48abcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5cb111df3ec4249a836518a598063a4, user_id=8ee5364270094596b64b08a8ac56f6ce, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:27.316776 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2a68b-b3c2-4354-89ff-cc9bad48abcc admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:27.318434 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e998470-214e-4e6a-8734-1a319825e06c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6183eeb3ff43e79eccb5d7ad08c53c, user_id=bd0929f371e4442eb4eb6e12a96bbfc4, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:27.321173 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e998470-214e-4e6a-8734-1a319825e06c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:27.339150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 799/1607] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:27 2022] PUT /identity/v3/projects/b5cb111df3ec4249a836518a598063a4/users/8ee5364270094596b64b08a8ac56f6ce/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:27.343981 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2a68b-b3c2-4354-89ff-cc9bad48abcc admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:27.347793 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 809/1608] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:27 2022] PUT /identity/v3/projects/7e6183eeb3ff43e79eccb5d7ad08c53c/users/bd0929f371e4442eb4eb6e12a96bbfc4/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:27.355921 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e3a42-db39-4db9-a95b-818976f622e5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.356617 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e3a42-db39-4db9-a95b-818976f622e5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.357103 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e3a42-db39-4db9-a95b-818976f622e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.392223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-134e3a42-db39-4db9-a95b-818976f622e5 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:27.395683 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-134e3a42-db39-4db9-a95b-818976f622e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:27.434176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2a68b-b3c2-4354-89ff-cc9bad48abcc admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:27.490768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 810/1609] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:27.490768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77244e-afc6-432c-aca8-1ec824b3d763 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.490768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77244e-afc6-432c-aca8-1ec824b3d763 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.490768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77244e-afc6-432c-aca8-1ec824b3d763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.526152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-bf77244e-afc6-432c-aca8-1ec824b3d763 None None] MFA Rules not processed for user `db10227415804c2287ec4a60a6cbd2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:27.527697 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bf77244e-afc6-432c-aca8-1ec824b3d763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:27.586321 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757f24b-deaf-43fb-9b36-e50d3b83e4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:27.587077 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757f24b-deaf-43fb-9b36-e50d3b83e4a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.587427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757f24b-deaf-43fb-9b36-e50d3b83e4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.587766 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757f24b-deaf-43fb-9b36-e50d3b83e4a4 admin admin] PATH_INFO: `/v3/projects/c0ae1cb6516648f2a11b19d6e4962b95/users/1b9dc331cec142288ec461e05becada3/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.609812 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1757f24b-deaf-43fb-9b36-e50d3b83e4a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ae1cb6516648f2a11b19d6e4962b95, user_id=1b9dc331cec142288ec461e05becada3, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:27.613945 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1757f24b-deaf-43fb-9b36-e50d3b83e4a4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:27.617368 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:27.630162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:27.645315 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 800/1610] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:27 2022] PUT /identity/v3/projects/c0ae1cb6516648f2a11b19d6e4962b95/users/1b9dc331cec142288ec461e05becada3/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:27.656211 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632cb29-d26c-4b78-b2e6-f0cefd780a00 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.656505 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632cb29-d26c-4b78-b2e6-f0cefd780a00 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.657696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632cb29-d26c-4b78-b2e6-f0cefd780a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.684056 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 811/1611] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:27.686789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c100c-ce58-4c14-83c7-c5bb0c038c50 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.688930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c100c-ce58-4c14-83c7-c5bb0c038c50 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.699008 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c100c-ce58-4c14-83c7-c5bb0c038c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.705225 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-4632cb29-d26c-4b78-b2e6-f0cefd780a00 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:27.707232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4632cb29-d26c-4b78-b2e6-f0cefd780a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:27.749123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-636c100c-ce58-4c14-83c7-c5bb0c038c50 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:27.750598 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-636c100c-ce58-4c14-83c7-c5bb0c038c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:27.782403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 801/1612] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:27.783580 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c917fc9-6166-4e79-ac45-78941932c8fa None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.783953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c917fc9-6166-4e79-ac45-78941932c8fa None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.784323 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c917fc9-6166-4e79-ac45-78941932c8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.804685 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7c917fc9-6166-4e79-ac45-78941932c8fa None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:27.805952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7c917fc9-6166-4e79-ac45-78941932c8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:27.815796 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 802/1613] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:27.817228 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad99ff-6f39-44c1-a919-0811a701b98f None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.817575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad99ff-6f39-44c1-a919-0811a701b98f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.817916 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad99ff-6f39-44c1-a919-0811a701b98f None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.819669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 803/1614] 10.222.0.67 () {54 vars in 912 bytes} [Wed Aug 24 08:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:55:27.820442 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad99ff-6f39-44c1-a919-0811a701b98f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:27.822174 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-bcad99ff-6f39-44c1-a919-0811a701b98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:27.858165 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 812/1615] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:27.862018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a7a99-37f0-4c7b-81b5-df53848b6e36 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.862018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a7a99-37f0-4c7b-81b5-df53848b6e36 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.862018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a7a99-37f0-4c7b-81b5-df53848b6e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.900123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-818a7a99-37f0-4c7b-81b5-df53848b6e36 None None] MFA Rules not processed for user `1b9dc331cec142288ec461e05becada3`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:27.902506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-818a7a99-37f0-4c7b-81b5-df53848b6e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:27.904582 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad99ff-6f39-44c1-a919-0811a701b98f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:27.908707 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0a1c8-ee6e-4f9c-9362-27abdd31c174 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:27.909421 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0a1c8-ee6e-4f9c-9362-27abdd31c174 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.909703 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0a1c8-ee6e-4f9c-9362-27abdd31c174 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.909975 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0a1c8-ee6e-4f9c-9362-27abdd31c174 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.925088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0a1c8-ee6e-4f9c-9362-27abdd31c174 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:27.927041 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0a1c8-ee6e-4f9c-9362-27abdd31c174 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:27.940496 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 804/1616] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:27.941813 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bcf94-8f49-44dd-819e-92e9f7dfe010 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.942100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bcf94-8f49-44dd-819e-92e9f7dfe010 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.942379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bcf94-8f49-44dd-819e-92e9f7dfe010 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.944912 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 805/1617] 10.222.0.67 () {54 vars in 962 bytes} [Wed Aug 24 08:55:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:55:27.945689 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687bcf94-8f49-44dd-819e-92e9f7dfe010 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:27.961896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687bcf94-8f49-44dd-819e-92e9f7dfe010 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:27.968342 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da0d94-a14f-4d15-82f6-1956ecc20a38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:27.968972 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da0d94-a14f-4d15-82f6-1956ecc20a38 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:27.969256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da0d94-a14f-4d15-82f6-1956ecc20a38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:27.969529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da0d94-a14f-4d15-82f6-1956ecc20a38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:27.988388 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da0d94-a14f-4d15-82f6-1956ecc20a38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:27.990564 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da0d94-a14f-4d15-82f6-1956ecc20a38 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:28.005669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 806/1618] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:28.006277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 813/1619] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:28.007417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a7a99-37f0-4c7b-81b5-df53848b6e36 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:28.010872 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f72c1-e924-4d2d-a065-f5579c9c9dcc None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:28.011256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f72c1-e924-4d2d-a065-f5579c9c9dcc None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:28.011624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f72c1-e924-4d2d-a065-f5579c9c9dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:28.019491 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a7a99-37f0-4c7b-81b5-df53848b6e36 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:28.026092 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423d712-cfed-4f19-83b4-1fd13371e3e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:28.026924 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423d712-cfed-4f19-83b4-1fd13371e3e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:28.027376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423d712-cfed-4f19-83b4-1fd13371e3e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:28.027819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423d712-cfed-4f19-83b4-1fd13371e3e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:28.037544 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-191f72c1-e924-4d2d-a065-f5579c9c9dcc None None] MFA Rules not processed for user `ca56dd7216fb4946946b2801e144fec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:28.038920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-191f72c1-e924-4d2d-a065-f5579c9c9dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:28.045864 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423d712-cfed-4f19-83b4-1fd13371e3e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:28.047843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423d712-cfed-4f19-83b4-1fd13371e3e5 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:28.063857 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 814/1620] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:28.067717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423d712-cfed-4f19-83b4-1fd13371e3e5 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:28.080900 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423d712-cfed-4f19-83b4-1fd13371e3e5 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:28.085382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ade31-8eb5-4a73-993e-4fc4b7842068 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:28.086025 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ade31-8eb5-4a73-993e-4fc4b7842068 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:28.086493 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ade31-8eb5-4a73-993e-4fc4b7842068 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:28.086975 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ade31-8eb5-4a73-993e-4fc4b7842068 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:28.102638 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ade31-8eb5-4a73-993e-4fc4b7842068 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:28.104325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ade31-8eb5-4a73-993e-4fc4b7842068 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:28.131846 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 815/1621] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:28.133439 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc112b9-7fb3-4477-bbd3-990ff4a71fa0 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:28.133724 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc112b9-7fb3-4477-bbd3-990ff4a71fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:28.133997 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc112b9-7fb3-4477-bbd3-990ff4a71fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:28.149581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 807/1622] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 24 08:55:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:28.150601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f72c1-e924-4d2d-a065-f5579c9c9dcc None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:28.176704 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f72c1-e924-4d2d-a065-f5579c9c9dcc None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:28.183384 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7809c3-7f6b-4006-a9df-8cca079c3a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:28.183807 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7809c3-7f6b-4006-a9df-8cca079c3a04 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:28.183990 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3cc112b9-7fb3-4477-bbd3-990ff4a71fa0 None None] MFA Rules not processed for user `ca56dd7216fb4946946b2801e144fec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:28.184292 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7809c3-7f6b-4006-a9df-8cca079c3a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:28.184451 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7809c3-7f6b-4006-a9df-8cca079c3a04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:28.185158 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7809c3-7f6b-4006-a9df-8cca079c3a04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:28.188666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7809c3-7f6b-4006-a9df-8cca079c3a04 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:28.191516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3cc112b9-7fb3-4477-bbd3-990ff4a71fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:28.196595 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:28.213038 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 816/1623] 10.222.0.67 () {58 vars in 1059 bytes} [Wed Aug 24 08:55:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:28.214143 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc112b9-7fb3-4477-bbd3-990ff4a71fa0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:28.242067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 808/1624] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:28 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:28.243802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7809c3-7f6b-4006-a9df-8cca079c3a04 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:28.287282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:28.318193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc112b9-7fb3-4477-bbd3-990ff4a71fa0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:28.360409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7809c3-7f6b-4006-a9df-8cca079c3a04 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:28.367308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164a636-73ed-45b8-a258-278ab59aab87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:28.368073 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164a636-73ed-45b8-a258-278ab59aab87 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:28.368489 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164a636-73ed-45b8-a258-278ab59aab87 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:28.368861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164a636-73ed-45b8-a258-278ab59aab87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:28.432085 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cff62c-ee0c-43b2-ae8d-6fa7fc93fa34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:28.433117 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cff62c-ee0c-43b2-ae8d-6fa7fc93fa34 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:28.433552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cff62c-ee0c-43b2-ae8d-6fa7fc93fa34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:28.434020 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cff62c-ee0c-43b2-ae8d-6fa7fc93fa34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:28.454719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cff62c-ee0c-43b2-ae8d-6fa7fc93fa34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:28.456720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cff62c-ee0c-43b2-ae8d-6fa7fc93fa34 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:28.479721 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 817/1625] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:28.481608 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cff62c-ee0c-43b2-ae8d-6fa7fc93fa34 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:28.485443 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164a636-73ed-45b8-a258-278ab59aab87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:28.487064 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164a636-73ed-45b8-a258-278ab59aab87 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:28.520548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 809/1626] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:28.651397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cff62c-ee0c-43b2-ae8d-6fa7fc93fa34 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:28.667093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c193f35-abda-49c6-b2f1-ff22bbb177e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:28.667734 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c193f35-abda-49c6-b2f1-ff22bbb177e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:28.668951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c193f35-abda-49c6-b2f1-ff22bbb177e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:28.672218 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c193f35-abda-49c6-b2f1-ff22bbb177e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:28.673024 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c193f35-abda-49c6-b2f1-ff22bbb177e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:28.679645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c193f35-abda-49c6-b2f1-ff22bbb177e7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:28.730648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 818/1627] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:28 2022] POST /identity/v3/users => generated 289 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:28.740221 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164a636-73ed-45b8-a258-278ab59aab87 service nova] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:28.772369 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164a636-73ed-45b8-a258-278ab59aab87 service nova] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:28.785309 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488fe7dd-116f-4eba-a987-43e2315f1407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:28.785309 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fe7dd-116f-4eba-a987-43e2315f1407 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:28.785309 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fe7dd-116f-4eba-a987-43e2315f1407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:28.785309 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fe7dd-116f-4eba-a987-43e2315f1407 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:28.785976 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488fe7dd-116f-4eba-a987-43e2315f1407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:28.787502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488fe7dd-116f-4eba-a987-43e2315f1407 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:28.815925 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 810/1628] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:28.819542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c193f35-abda-49c6-b2f1-ff22bbb177e7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:28.839648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c193f35-abda-49c6-b2f1-ff22bbb177e7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:28.855148 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e164f-3737-4627-ad06-5a62b7bbe591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:28.855817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e164f-3737-4627-ad06-5a62b7bbe591 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:28.856142 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e164f-3737-4627-ad06-5a62b7bbe591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:28.856427 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e164f-3737-4627-ad06-5a62b7bbe591 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:28.857135 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e164f-3737-4627-ad06-5a62b7bbe591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:28.861711 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e164f-3737-4627-ad06-5a62b7bbe591 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:28.898627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 819/1629] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:28.900124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488fe7dd-116f-4eba-a987-43e2315f1407 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:28.915959 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488fe7dd-116f-4eba-a987-43e2315f1407 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:28.941846 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623c1b1-548d-4c0c-8e9e-7f469bcaca18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:28.942686 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623c1b1-548d-4c0c-8e9e-7f469bcaca18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:28.943433 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623c1b1-548d-4c0c-8e9e-7f469bcaca18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:28.943815 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623c1b1-548d-4c0c-8e9e-7f469bcaca18 admin admin] PATH_INFO: `/v3/projects/b56f14e1e9074c7185527a9f78b144b6/users/1d29c1a15b194f45bb72ad2d3bfe3ef1/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:28.981703 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b5113-1c72-4dd4-9623-4bc8213a1da7 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:28.982263 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b5113-1c72-4dd4-9623-4bc8213a1da7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:28.985086 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b5113-1c72-4dd4-9623-4bc8213a1da7 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:28.986054 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623c1b1-548d-4c0c-8e9e-7f469bcaca18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b56f14e1e9074c7185527a9f78b144b6, user_id=1d29c1a15b194f45bb72ad2d3bfe3ef1, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:28.986476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 820/1630] 10.222.0.67 () {54 vars in 912 bytes} [Wed Aug 24 08:55:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:55:28.992202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623c1b1-548d-4c0c-8e9e-7f469bcaca18 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:28.995644 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf44205-9b07-4d89-b058-f3b70b6ee561 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:28.996072 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf44205-9b07-4d89-b058-f3b70b6ee561 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:28.996578 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf44205-9b07-4d89-b058-f3b70b6ee561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:29.032432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 811/1631] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:28 2022] PUT /identity/v3/projects/b56f14e1e9074c7185527a9f78b144b6/users/1d29c1a15b194f45bb72ad2d3bfe3ef1/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:29.032650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-aaf44205-9b07-4d89-b058-f3b70b6ee561 None None] MFA Rules not processed for user `ca56dd7216fb4946946b2801e144fec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:29.033474 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-aaf44205-9b07-4d89-b058-f3b70b6ee561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:29.039589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80a5a8-7371-49e0-ab07-044756cf90cc None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:29.045004 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80a5a8-7371-49e0-ab07-044756cf90cc None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:29.046082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80a5a8-7371-49e0-ab07-044756cf90cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:29.183153 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-3c80a5a8-7371-49e0-ab07-044756cf90cc None None] MFA Rules not processed for user `1d29c1a15b194f45bb72ad2d3bfe3ef1`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:29.184363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3c80a5a8-7371-49e0-ab07-044756cf90cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:29.223730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 821/1632] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 24 08:55:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:55:29.247959 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf44205-9b07-4d89-b058-f3b70b6ee561 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:29.267972 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:29.304080 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:29.339152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf44205-9b07-4d89-b058-f3b70b6ee561 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:29.357101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f97b1-84f4-4cf7-bead-ead1acb7b7d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:29.357736 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f97b1-84f4-4cf7-bead-ead1acb7b7d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:29.359113 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f97b1-84f4-4cf7-bead-ead1acb7b7d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:29.359422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f97b1-84f4-4cf7-bead-ead1acb7b7d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:29.376219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 812/1633] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:55:29.379983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80a5a8-7371-49e0-ab07-044756cf90cc None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:29.410962 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f97b1-84f4-4cf7-bead-ead1acb7b7d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:29.418982 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f97b1-84f4-4cf7-bead-ead1acb7b7d6 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:29.486560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 822/1634] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:29.583137 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80a5a8-7371-49e0-ab07-044756cf90cc None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:29.588906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8668bf8-7adc-42d1-a530-1558dbe84d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:29.589545 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8668bf8-7adc-42d1-a530-1558dbe84d15 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:29.589919 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8668bf8-7adc-42d1-a530-1558dbe84d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:29.590197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8668bf8-7adc-42d1-a530-1558dbe84d15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:29.590990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8668bf8-7adc-42d1-a530-1558dbe84d15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:29.595486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8668bf8-7adc-42d1-a530-1558dbe84d15 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:29.645918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 813/1635] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:29 2022] POST /identity/v3/projects => generated 377 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:29.650461 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f97b1-84f4-4cf7-bead-ead1acb7b7d6 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:29.668993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f97b1-84f4-4cf7-bead-ead1acb7b7d6 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:29.778918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d14f5-0925-4f4a-9c9d-f5a569a1ce43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:29.779951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d14f5-0925-4f4a-9c9d-f5a569a1ce43 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:29.780531 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d14f5-0925-4f4a-9c9d-f5a569a1ce43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:29.781197 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d14f5-0925-4f4a-9c9d-f5a569a1ce43 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:29.781871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d14f5-0925-4f4a-9c9d-f5a569a1ce43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:29.784126 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d14f5-0925-4f4a-9c9d-f5a569a1ce43 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:29.824848 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 823/1636] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:29 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:29.826420 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8668bf8-7adc-42d1-a530-1558dbe84d15 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:29.840836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8668bf8-7adc-42d1-a530-1558dbe84d15 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:29.846764 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc0f4e-6d9a-4ced-b4a9-03c61032265e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:29.847294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc0f4e-6d9a-4ced-b4a9-03c61032265e admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:29.847623 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc0f4e-6d9a-4ced-b4a9-03c61032265e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:29.847896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc0f4e-6d9a-4ced-b4a9-03c61032265e admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:29.848715 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc0f4e-6d9a-4ced-b4a9-03c61032265e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:29.850682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc0f4e-6d9a-4ced-b4a9-03c61032265e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:29.866425 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 814/1637] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:29.870791 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d14f5-0925-4f4a-9c9d-f5a569a1ce43 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:29.891352 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d14f5-0925-4f4a-9c9d-f5a569a1ce43 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:29.904561 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf1587-282e-43dd-ad5c-fdd918e71708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:29.909676 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf1587-282e-43dd-ad5c-fdd918e71708 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:29.910204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf1587-282e-43dd-ad5c-fdd918e71708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:29.910941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf1587-282e-43dd-ad5c-fdd918e71708 admin admin] PATH_INFO: `/v3/projects/9bda613af4384e8d841fe0f800aff5a1/users/de7603c42c5042a5a6d717a47ec5a65c/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:29.955974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bf1587-282e-43dd-ad5c-fdd918e71708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bda613af4384e8d841fe0f800aff5a1, user_id=de7603c42c5042a5a6d717a47ec5a65c, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:29.961923 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bf1587-282e-43dd-ad5c-fdd918e71708 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:30.000177 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 824/1638] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:29 2022] PUT /identity/v3/projects/9bda613af4384e8d841fe0f800aff5a1/users/de7603c42c5042a5a6d717a47ec5a65c/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:55:30.005097 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc0f4e-6d9a-4ced-b4a9-03c61032265e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:30.047917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc0f4e-6d9a-4ced-b4a9-03c61032265e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:30.190629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547e24e-6af2-4089-b6bd-8770984fca73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:30.191257 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547e24e-6af2-4089-b6bd-8770984fca73 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:30.191535 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547e24e-6af2-4089-b6bd-8770984fca73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:30.191803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547e24e-6af2-4089-b6bd-8770984fca73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:30.192689 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547e24e-6af2-4089-b6bd-8770984fca73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:30.194379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547e24e-6af2-4089-b6bd-8770984fca73 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:30.209495 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 815/1639] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:30.213879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf1587-282e-43dd-ad5c-fdd918e71708 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:30.230706 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf1587-282e-43dd-ad5c-fdd918e71708 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:30.238978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42426456-659f-49e9-aae4-f5dc2cafc4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:30.240028 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42426456-659f-49e9-aae4-f5dc2cafc4de admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:30.240639 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42426456-659f-49e9-aae4-f5dc2cafc4de admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:30.241070 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42426456-659f-49e9-aae4-f5dc2cafc4de admin admin] PATH_INFO: `/v3/projects/9bda613af4384e8d841fe0f800aff5a1/users/de7603c42c5042a5a6d717a47ec5a65c/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:30.243877 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42426456-659f-49e9-aae4-f5dc2cafc4de admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bda613af4384e8d841fe0f800aff5a1, user_id=de7603c42c5042a5a6d717a47ec5a65c, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:30.248120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42426456-659f-49e9-aae4-f5dc2cafc4de admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:30.266885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 825/1640] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:30 2022] PUT /identity/v3/projects/9bda613af4384e8d841fe0f800aff5a1/users/de7603c42c5042a5a6d717a47ec5a65c/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:30.277430 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573a99f-0251-4c74-921f-3c31b6e3fb36 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:30.278914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573a99f-0251-4c74-921f-3c31b6e3fb36 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:30.278914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573a99f-0251-4c74-921f-3c31b6e3fb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:30.318131 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1573a99f-0251-4c74-921f-3c31b6e3fb36 None None] MFA Rules not processed for user `de7603c42c5042a5a6d717a47ec5a65c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:30.319793 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1573a99f-0251-4c74-921f-3c31b6e3fb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:30.388833 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:30.500151 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 816/1641] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:30.518812 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42426456-659f-49e9-aae4-f5dc2cafc4de admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:30.634052 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42426456-659f-49e9-aae4-f5dc2cafc4de admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:30.646468 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf150dbc-619e-46b7-a84b-4fab4cb9ad4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:30.648379 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf150dbc-619e-46b7-a84b-4fab4cb9ad4e service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:30.650145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf150dbc-619e-46b7-a84b-4fab4cb9ad4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:30.650859 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf150dbc-619e-46b7-a84b-4fab4cb9ad4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:30.680214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf150dbc-619e-46b7-a84b-4fab4cb9ad4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:30.682009 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf150dbc-619e-46b7-a84b-4fab4cb9ad4e service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:30.696424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 826/1642] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:30.769682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573a99f-0251-4c74-921f-3c31b6e3fb36 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:30.787178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573a99f-0251-4c74-921f-3c31b6e3fb36 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:30.803811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134fcc2a-9719-4e1d-9d14-11703c547ee6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:30.805592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fcc2a-9719-4e1d-9d14-11703c547ee6 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:30.810328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fcc2a-9719-4e1d-9d14-11703c547ee6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:30.811804 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134fcc2a-9719-4e1d-9d14-11703c547ee6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:30.974979 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134fcc2a-9719-4e1d-9d14-11703c547ee6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:30.976758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134fcc2a-9719-4e1d-9d14-11703c547ee6 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:30.995947 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 817/1643] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:33.679663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf150dbc-619e-46b7-a84b-4fab4cb9ad4e service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:33.704258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:33.787392 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf150dbc-619e-46b7-a84b-4fab4cb9ad4e service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:33.796776 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bce3e4-26b1-4294-9a5c-20abfa00cf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:33.797474 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bce3e4-26b1-4294-9a5c-20abfa00cf79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:33.797754 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bce3e4-26b1-4294-9a5c-20abfa00cf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:33.798029 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bce3e4-26b1-4294-9a5c-20abfa00cf79 admin admin] PATH_INFO: `/v3/users/e84cbb1c1968413989870678d1165d17` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:33.799839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bce3e4-26b1-4294-9a5c-20abfa00cf79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e84cbb1c1968413989870678d1165d17)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:33.802225 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bce3e4-26b1-4294-9a5c-20abfa00cf79 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:33.876795 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-53bce3e4-26b1-4294-9a5c-20abfa00cf79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e84cbb1c1968413989870678d1165d17'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:33.877622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-53bce3e4-26b1-4294-9a5c-20abfa00cf79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e84cbb1c1968413989870678d1165d17'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:33.888619 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-53bce3e4-26b1-4294-9a5c-20abfa00cf79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e84cbb1c1968413989870678d1165d17'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:33.916329 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-53bce3e4-26b1-4294-9a5c-20abfa00cf79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e84cbb1c1968413989870678d1165d17'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:33.957524 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-53bce3e4-26b1-4294-9a5c-20abfa00cf79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e84cbb1c1968413989870678d1165d17'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:33.976845 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 827/1644] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:55:33 2022] DELETE /identity/v3/users/e84cbb1c1968413989870678d1165d17 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:33.981163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf4f59-438f-4d5b-a3b1-b0dd84a16a95 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:33.982068 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf4f59-438f-4d5b-a3b1-b0dd84a16a95 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:33.982488 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf4f59-438f-4d5b-a3b1-b0dd84a16a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:34.021492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-6edf4f59-438f-4d5b-a3b1-b0dd84a16a95 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:34.022725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6edf4f59-438f-4d5b-a3b1-b0dd84a16a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:34.147113 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 818/1645] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:34.182772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bce3e4-26b1-4294-9a5c-20abfa00cf79 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:34.185586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-53bce3e4-26b1-4294-9a5c-20abfa00cf79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:34.300304 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bce3e4-26b1-4294-9a5c-20abfa00cf79 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:34.306925 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965511c-e3fa-43ab-bf76-92af4cfb1337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:34.308369 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965511c-e3fa-43ab-bf76-92af4cfb1337 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:34.308968 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965511c-e3fa-43ab-bf76-92af4cfb1337 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:34.309446 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965511c-e3fa-43ab-bf76-92af4cfb1337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:34.331894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e965511c-e3fa-43ab-bf76-92af4cfb1337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:34.334250 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e965511c-e3fa-43ab-bf76-92af4cfb1337 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:34.357086 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 828/1646] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:35.752497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf4f59-438f-4d5b-a3b1-b0dd84a16a95 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:35.756714 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6edf4f59-438f-4d5b-a3b1-b0dd84a16a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:35.802556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf4f59-438f-4d5b-a3b1-b0dd84a16a95 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:35.826241 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf266335-c235-45f1-94c9-581fc4367843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:35.832387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf266335-c235-45f1-94c9-581fc4367843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:35.833313 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf266335-c235-45f1-94c9-581fc4367843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:35.833985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf266335-c235-45f1-94c9-581fc4367843 admin admin] PATH_INFO: `/v3/projects/dba44ee9cc0c427a81b9c539b056e30d` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:35.849329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf266335-c235-45f1-94c9-581fc4367843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dba44ee9cc0c427a81b9c539b056e30d)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:35.857944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf266335-c235-45f1-94c9-581fc4367843 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:36.029318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bf266335-c235-45f1-94c9-581fc4367843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dba44ee9cc0c427a81b9c539b056e30d', 'request_id': 'req-bf266335-c235-45f1-94c9-581fc4367843'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:36.073858 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bf266335-c235-45f1-94c9-581fc4367843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dba44ee9cc0c427a81b9c539b056e30d', 'request_id': 'req-bf266335-c235-45f1-94c9-581fc4367843'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:36.092028 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bf266335-c235-45f1-94c9-581fc4367843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dba44ee9cc0c427a81b9c539b056e30d', 'request_id': 'req-bf266335-c235-45f1-94c9-581fc4367843'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:36.097657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:36.117750 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bf266335-c235-45f1-94c9-581fc4367843 admin admin] The token cache is being invalidate because project dba44ee9cc0c427a81b9c539b056e30d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:55:36.118046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bf266335-c235-45f1-94c9-581fc4367843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:36.123131 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bf266335-c235-45f1-94c9-581fc4367843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:36.129735 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 819/1647] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:55:35 2022] DELETE /identity/v3/projects/dba44ee9cc0c427a81b9c539b056e30d => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:36.130087 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eaf6d2-acc6-4e61-9d26-96af83a353c3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:36.130562 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eaf6d2-acc6-4e61-9d26-96af83a353c3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:36.130562 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eaf6d2-acc6-4e61-9d26-96af83a353c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:36.265656 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f5eaf6d2-acc6-4e61-9d26-96af83a353c3 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:36.275021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f5eaf6d2-acc6-4e61-9d26-96af83a353c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:36.424495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:36.595733 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 829/1648] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:36.597727 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43956c88-96fb-4d57-b2c5-e29aecba400c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:36.598132 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43956c88-96fb-4d57-b2c5-e29aecba400c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:36.598547 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43956c88-96fb-4d57-b2c5-e29aecba400c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:36.637574 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-43956c88-96fb-4d57-b2c5-e29aecba400c None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:36.639031 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-43956c88-96fb-4d57-b2c5-e29aecba400c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:36.688177 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eaf6d2-acc6-4e61-9d26-96af83a353c3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:36.688787 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 820/1649] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:36.698996 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eaf6d2-acc6-4e61-9d26-96af83a353c3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:36.722903 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15383f-6abd-40ff-998a-8f5e763df55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:36.723797 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15383f-6abd-40ff-998a-8f5e763df55e admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:36.723953 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15383f-6abd-40ff-998a-8f5e763df55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:36.724269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15383f-6abd-40ff-998a-8f5e763df55e admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:36.724944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15383f-6abd-40ff-998a-8f5e763df55e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:36.725248 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15383f-6abd-40ff-998a-8f5e763df55e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:36.727777 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15383f-6abd-40ff-998a-8f5e763df55e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:36.785076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 830/1650] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:55:36.788321 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43956c88-96fb-4d57-b2c5-e29aecba400c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:36.825686 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43956c88-96fb-4d57-b2c5-e29aecba400c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:36.855762 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7330b-ac2e-4ab2-bd30-ed2f01fac309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:36.858007 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7330b-ac2e-4ab2-bd30-ed2f01fac309 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:36.858529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7330b-ac2e-4ab2-bd30-ed2f01fac309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:36.858930 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7330b-ac2e-4ab2-bd30-ed2f01fac309 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:36.860847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7330b-ac2e-4ab2-bd30-ed2f01fac309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:36.862797 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7330b-ac2e-4ab2-bd30-ed2f01fac309 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:36.965959 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 821/1651] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:36 2022] POST /identity/v3/projects => generated 365 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:36.968584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15383f-6abd-40ff-998a-8f5e763df55e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:36.995663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15383f-6abd-40ff-998a-8f5e763df55e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:37.138645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0b517-68eb-428e-b1ff-0be8fefd3190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:37.141533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0b517-68eb-428e-b1ff-0be8fefd3190 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:37.142796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0b517-68eb-428e-b1ff-0be8fefd3190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:37.143291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0b517-68eb-428e-b1ff-0be8fefd3190 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:37.144199 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0b517-68eb-428e-b1ff-0be8fefd3190 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:37.150707 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0b517-68eb-428e-b1ff-0be8fefd3190 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:37.216803 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 831/1652] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:36 2022] POST /identity/v3/users => generated 283 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:37.218045 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7330b-ac2e-4ab2-bd30-ed2f01fac309 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:37.257236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7330b-ac2e-4ab2-bd30-ed2f01fac309 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:37.291785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b3119-e1d5-4007-b70b-66cd3f612bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:37.294227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b3119-e1d5-4007-b70b-66cd3f612bbd admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:37.294912 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b3119-e1d5-4007-b70b-66cd3f612bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:37.298944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b3119-e1d5-4007-b70b-66cd3f612bbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:37.300828 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b3119-e1d5-4007-b70b-66cd3f612bbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:37.307499 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b3119-e1d5-4007-b70b-66cd3f612bbd admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:37.330699 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 822/1653] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:37.335275 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0b517-68eb-428e-b1ff-0be8fefd3190 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:37.354687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0b517-68eb-428e-b1ff-0be8fefd3190 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:37.381436 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92567cf-a521-4636-9f70-2d3e5e122d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:37.384029 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92567cf-a521-4636-9f70-2d3e5e122d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:37.384751 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92567cf-a521-4636-9f70-2d3e5e122d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:37.385126 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92567cf-a521-4636-9f70-2d3e5e122d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:37.385919 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92567cf-a521-4636-9f70-2d3e5e122d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:37.387666 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92567cf-a521-4636-9f70-2d3e5e122d45 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:37.406513 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 832/1654] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:37.413234 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b3119-e1d5-4007-b70b-66cd3f612bbd admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:37.428560 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b3119-e1d5-4007-b70b-66cd3f612bbd admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:37.453791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442d061-cc51-4b7c-ae0a-490d1a0a25df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:37.455427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442d061-cc51-4b7c-ae0a-490d1a0a25df admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:37.456018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442d061-cc51-4b7c-ae0a-490d1a0a25df admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:37.457337 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442d061-cc51-4b7c-ae0a-490d1a0a25df admin admin] PATH_INFO: `/v3/projects/3daf165199e2404d98b908d32941e72a/users/f0c2b7f3f63f4b5395999af532060adf/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:37.509950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442d061-cc51-4b7c-ae0a-490d1a0a25df admin admin] RBAC: Authorizing `identity:create_grant(project_id=3daf165199e2404d98b908d32941e72a, user_id=f0c2b7f3f63f4b5395999af532060adf, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:37.513853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442d061-cc51-4b7c-ae0a-490d1a0a25df admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:37.574823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 823/1655] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:37 2022] PUT /identity/v3/projects/3daf165199e2404d98b908d32941e72a/users/f0c2b7f3f63f4b5395999af532060adf/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:37.577051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fdba3-145f-4ddb-832c-19316ce20970 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:37.577546 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fdba3-145f-4ddb-832c-19316ce20970 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:37.577918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fdba3-145f-4ddb-832c-19316ce20970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:37.647600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-af6fdba3-145f-4ddb-832c-19316ce20970 None None] MFA Rules not processed for user `f0c2b7f3f63f4b5395999af532060adf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:37.651131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-af6fdba3-145f-4ddb-832c-19316ce20970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:37.981879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 833/1656] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:37.995969 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442d061-cc51-4b7c-ae0a-490d1a0a25df admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:37.997889 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c442d061-cc51-4b7c-ae0a-490d1a0a25df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:38.214442 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442d061-cc51-4b7c-ae0a-490d1a0a25df admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:38.230363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16adae-b278-4591-87e6-0c7564cd1b51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:38.231018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16adae-b278-4591-87e6-0c7564cd1b51 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:38.231294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16adae-b278-4591-87e6-0c7564cd1b51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:38.231564 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16adae-b278-4591-87e6-0c7564cd1b51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:38.263795 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16adae-b278-4591-87e6-0c7564cd1b51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:38.266663 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16adae-b278-4591-87e6-0c7564cd1b51 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:38.293372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 824/1657] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:39.698433 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67e903-3d2b-4cc4-ae88-cf226115a27a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:39.698902 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67e903-3d2b-4cc4-ae88-cf226115a27a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:39.699496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67e903-3d2b-4cc4-ae88-cf226115a27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:39.852273 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-af67e903-3d2b-4cc4-ae88-cf226115a27a None None] MFA Rules not processed for user `f0dc2dcc5327489c9ecc28a43ab7c0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:39.855673 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-af67e903-3d2b-4cc4-ae88-cf226115a27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:39.917553 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:40.017006 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16adae-b278-4591-87e6-0c7564cd1b51 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:40.019203 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-be16adae-b278-4591-87e6-0c7564cd1b51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:40.130507 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 834/1658] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:40.150581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67e903-3d2b-4cc4-ae88-cf226115a27a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:40.152074 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-af67e903-3d2b-4cc4-ae88-cf226115a27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:40.259148 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67e903-3d2b-4cc4-ae88-cf226115a27a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:40.266246 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d667a8b-f61d-45fe-808d-fb9244d9b420 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:40.267063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d667a8b-f61d-45fe-808d-fb9244d9b420 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:40.267350 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d667a8b-f61d-45fe-808d-fb9244d9b420 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:40.267621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d667a8b-f61d-45fe-808d-fb9244d9b420 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:40.277174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d667a8b-f61d-45fe-808d-fb9244d9b420 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:40.278856 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d667a8b-f61d-45fe-808d-fb9244d9b420 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:40.292535 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 835/1659] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:55:40.331872 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16adae-b278-4591-87e6-0c7564cd1b51 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:40.352574 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfb4a7-5021-4ad4-97a9-36210a24a29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:40.353690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfb4a7-5021-4ad4-97a9-36210a24a29a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:40.354156 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfb4a7-5021-4ad4-97a9-36210a24a29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:40.354577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfb4a7-5021-4ad4-97a9-36210a24a29a admin admin] PATH_INFO: `/v3/users/1b9dc331cec142288ec461e05becada3` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:40.356613 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dfb4a7-5021-4ad4-97a9-36210a24a29a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b9dc331cec142288ec461e05becada3)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:40.360647 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: WARNING py.warnings [None req-e3dfb4a7-5021-4ad4-97a9-36210a24a29a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:55:40.360647 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: warnings.warn(msg) Aug 24 08:55:40.360647 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: Aug 24 08:55:40.362474 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dfb4a7-5021-4ad4-97a9-36210a24a29a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:40.556496 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:40.585585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e3dfb4a7-5021-4ad4-97a9-36210a24a29a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b9dc331cec142288ec461e05becada3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:40.599248 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e3dfb4a7-5021-4ad4-97a9-36210a24a29a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b9dc331cec142288ec461e05becada3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:40.651264 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e3dfb4a7-5021-4ad4-97a9-36210a24a29a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b9dc331cec142288ec461e05becada3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:40.686113 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e3dfb4a7-5021-4ad4-97a9-36210a24a29a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b9dc331cec142288ec461e05becada3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:40.688301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e3dfb4a7-5021-4ad4-97a9-36210a24a29a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b9dc331cec142288ec461e05becada3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:40.701054 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 825/1660] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:55:40 2022] DELETE /identity/v3/users/1b9dc331cec142288ec461e05becada3 => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:40.707952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd6d77-b691-458c-bd99-0ecbb7783ff5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:40.708532 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd6d77-b691-458c-bd99-0ecbb7783ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:40.708738 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd6d77-b691-458c-bd99-0ecbb7783ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:40.768439 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5acd6d77-b691-458c-bd99-0ecbb7783ff5 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:40.772653 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5acd6d77-b691-458c-bd99-0ecbb7783ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:40.992333 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 836/1661] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:41.168312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfb4a7-5021-4ad4-97a9-36210a24a29a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:41.170820 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e3dfb4a7-5021-4ad4-97a9-36210a24a29a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:41.364011 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfb4a7-5021-4ad4-97a9-36210a24a29a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:41.369642 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99661003-4056-4f82-8544-ca50dcd7fc9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:41.370577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99661003-4056-4f82-8544-ca50dcd7fc9a service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:41.372184 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99661003-4056-4f82-8544-ca50dcd7fc9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:41.372477 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99661003-4056-4f82-8544-ca50dcd7fc9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:41.391137 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99661003-4056-4f82-8544-ca50dcd7fc9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:41.394099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99661003-4056-4f82-8544-ca50dcd7fc9a service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:41.417473 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 826/1662] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:42.490024 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd6d77-b691-458c-bd99-0ecbb7783ff5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:42.492314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5acd6d77-b691-458c-bd99-0ecbb7783ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:42.523263 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd6d77-b691-458c-bd99-0ecbb7783ff5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:42.534239 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e31475-bdd5-4f35-8c7c-393a667539a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:42.535285 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e31475-bdd5-4f35-8c7c-393a667539a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:42.535777 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e31475-bdd5-4f35-8c7c-393a667539a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:42.536299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e31475-bdd5-4f35-8c7c-393a667539a7 admin admin] PATH_INFO: `/v3/users/db10227415804c2287ec4a60a6cbd2d5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:42.538011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e31475-bdd5-4f35-8c7c-393a667539a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db10227415804c2287ec4a60a6cbd2d5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:42.540588 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e31475-bdd5-4f35-8c7c-393a667539a7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:42.643335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:42.655698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a7e31475-bdd5-4f35-8c7c-393a667539a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db10227415804c2287ec4a60a6cbd2d5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:42.657289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a7e31475-bdd5-4f35-8c7c-393a667539a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db10227415804c2287ec4a60a6cbd2d5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:42.677152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a7e31475-bdd5-4f35-8c7c-393a667539a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db10227415804c2287ec4a60a6cbd2d5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:42.692820 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a7e31475-bdd5-4f35-8c7c-393a667539a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db10227415804c2287ec4a60a6cbd2d5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:42.722654 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a7e31475-bdd5-4f35-8c7c-393a667539a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db10227415804c2287ec4a60a6cbd2d5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:42.736006 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 837/1663] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:55:42 2022] DELETE /identity/v3/users/db10227415804c2287ec4a60a6cbd2d5 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:42.745581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472e75e-8a10-4cf6-8b7b-70a62f2d6217 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:42.745976 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472e75e-8a10-4cf6-8b7b-70a62f2d6217 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:42.746073 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472e75e-8a10-4cf6-8b7b-70a62f2d6217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:42.790602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e472e75e-8a10-4cf6-8b7b-70a62f2d6217 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:42.792200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e472e75e-8a10-4cf6-8b7b-70a62f2d6217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:42.935291 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 827/1664] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:42.990670 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e31475-bdd5-4f35-8c7c-393a667539a7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:42.992230 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a7e31475-bdd5-4f35-8c7c-393a667539a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:43.143407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e31475-bdd5-4f35-8c7c-393a667539a7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:43.150371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4042c-970e-45e1-9926-1e2fc06201ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:43.151081 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4042c-970e-45e1-9926-1e2fc06201ab service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:43.151368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4042c-970e-45e1-9926-1e2fc06201ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:43.151644 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4042c-970e-45e1-9926-1e2fc06201ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:43.201067 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4042c-970e-45e1-9926-1e2fc06201ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:43.202764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4042c-970e-45e1-9926-1e2fc06201ab service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:43.230236 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 838/1665] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:55:43.729824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472e75e-8a10-4cf6-8b7b-70a62f2d6217 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:43.731464 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e472e75e-8a10-4cf6-8b7b-70a62f2d6217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:43.767631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472e75e-8a10-4cf6-8b7b-70a62f2d6217 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:43.774411 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f655e-5eff-4356-838c-d333c8fee8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:43.777509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f655e-5eff-4356-838c-d333c8fee8c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:43.778014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f655e-5eff-4356-838c-d333c8fee8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:43.778460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f655e-5eff-4356-838c-d333c8fee8c2 admin admin] PATH_INFO: `/v3/projects/c0ae1cb6516648f2a11b19d6e4962b95` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:43.779920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f655e-5eff-4356-838c-d333c8fee8c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ae1cb6516648f2a11b19d6e4962b95)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:43.786254 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f655e-5eff-4356-838c-d333c8fee8c2 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:43.934726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9ac6d-e4c7-44e3-a82a-98783aa9a91c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:43.935191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9ac6d-e4c7-44e3-a82a-98783aa9a91c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:43.935262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9ac6d-e4c7-44e3-a82a-98783aa9a91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:44.032916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3da9ac6d-e4c7-44e3-a82a-98783aa9a91c None None] MFA Rules not processed for user `f0d0eb9dc0f743c3985218901334b336`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:44.034237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3da9ac6d-e4c7-44e3-a82a-98783aa9a91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:44.035125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-cc3f655e-5eff-4356-838c-d333c8fee8c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ae1cb6516648f2a11b19d6e4962b95', 'request_id': 'req-cc3f655e-5eff-4356-838c-d333c8fee8c2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:44.043047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:44.060528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-cc3f655e-5eff-4356-838c-d333c8fee8c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ae1cb6516648f2a11b19d6e4962b95', 'request_id': 'req-cc3f655e-5eff-4356-838c-d333c8fee8c2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:44.073996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-cc3f655e-5eff-4356-838c-d333c8fee8c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ae1cb6516648f2a11b19d6e4962b95', 'request_id': 'req-cc3f655e-5eff-4356-838c-d333c8fee8c2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:44.101587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-cc3f655e-5eff-4356-838c-d333c8fee8c2 admin admin] The token cache is being invalidate because project c0ae1cb6516648f2a11b19d6e4962b95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:55:44.101964 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-cc3f655e-5eff-4356-838c-d333c8fee8c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:44.104373 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-cc3f655e-5eff-4356-838c-d333c8fee8c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:44.110039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 828/1666] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:55:43 2022] DELETE /identity/v3/projects/c0ae1cb6516648f2a11b19d6e4962b95 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:55:44.117390 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f655e-5eff-4356-838c-d333c8fee8c2 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:44.119895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-cc3f655e-5eff-4356-838c-d333c8fee8c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:44.291406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 839/1667] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:55:44.293262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695edf31-a6ca-4dd4-b6d8-8d85e5740cbc None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:44.293646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695edf31-a6ca-4dd4-b6d8-8d85e5740cbc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:44.294990 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695edf31-a6ca-4dd4-b6d8-8d85e5740cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:44.319108 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f655e-5eff-4356-838c-d333c8fee8c2 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:44.342843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4192aba-6ea2-4404-ba5d-7905ce40c2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:44.344168 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4192aba-6ea2-4404-ba5d-7905ce40c2a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:44.344633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4192aba-6ea2-4404-ba5d-7905ce40c2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:44.345147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4192aba-6ea2-4404-ba5d-7905ce40c2a9 admin admin] PATH_INFO: `/v3/users/1d29c1a15b194f45bb72ad2d3bfe3ef1` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:44.358909 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4192aba-6ea2-4404-ba5d-7905ce40c2a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d29c1a15b194f45bb72ad2d3bfe3ef1)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:44.389728 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4192aba-6ea2-4404-ba5d-7905ce40c2a9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:44.525439 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-695edf31-a6ca-4dd4-b6d8-8d85e5740cbc None None] MFA Rules not processed for user `8ee5364270094596b64b08a8ac56f6ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:44.536298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-695edf31-a6ca-4dd4-b6d8-8d85e5740cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:44.588697 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a4192aba-6ea2-4404-ba5d-7905ce40c2a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d29c1a15b194f45bb72ad2d3bfe3ef1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:44.609406 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a4192aba-6ea2-4404-ba5d-7905ce40c2a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d29c1a15b194f45bb72ad2d3bfe3ef1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:44.625827 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:44.665055 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a4192aba-6ea2-4404-ba5d-7905ce40c2a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d29c1a15b194f45bb72ad2d3bfe3ef1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:44.691695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a4192aba-6ea2-4404-ba5d-7905ce40c2a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d29c1a15b194f45bb72ad2d3bfe3ef1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:44.692581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a4192aba-6ea2-4404-ba5d-7905ce40c2a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d29c1a15b194f45bb72ad2d3bfe3ef1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:44.714568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 829/1668] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:55:44 2022] DELETE /identity/v3/users/1d29c1a15b194f45bb72ad2d3bfe3ef1 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:44.715604 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4192aba-6ea2-4404-ba5d-7905ce40c2a9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:44.719675 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a4192aba-6ea2-4404-ba5d-7905ce40c2a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:44.721502 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 840/1669] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:44.737413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1d496-1737-447b-8080-ff1f145ac84b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:44.741710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1d496-1737-447b-8080-ff1f145ac84b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:44.744329 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1d496-1737-447b-8080-ff1f145ac84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:44.782389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-aea1d496-1737-447b-8080-ff1f145ac84b None None] MFA Rules not processed for user `8ee5364270094596b64b08a8ac56f6ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:44.785508 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-aea1d496-1737-447b-8080-ff1f145ac84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:44.881022 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:44.913041 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 841/1670] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:44.928239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4192aba-6ea2-4404-ba5d-7905ce40c2a9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:44.947592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda446cf-bec3-42c7-add9-6e4af1c04620 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:44.947982 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda446cf-bec3-42c7-add9-6e4af1c04620 service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:44.948473 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda446cf-bec3-42c7-add9-6e4af1c04620 service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:44.948882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda446cf-bec3-42c7-add9-6e4af1c04620 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:44.951020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dda446cf-bec3-42c7-add9-6e4af1c04620 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:45.019658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1d496-1737-447b-8080-ff1f145ac84b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:45.021163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-aea1d496-1737-447b-8080-ff1f145ac84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:45.082279 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1d496-1737-447b-8080-ff1f145ac84b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:45.094930 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda446cf-bec3-42c7-add9-6e4af1c04620 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:45.096647 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda446cf-bec3-42c7-add9-6e4af1c04620 service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:45.113456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29af49-90ea-4c8e-842a-c93ab1f64769 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:45.114117 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29af49-90ea-4c8e-842a-c93ab1f64769 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:45.114403 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29af49-90ea-4c8e-842a-c93ab1f64769 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:45.114684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29af49-90ea-4c8e-842a-c93ab1f64769 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:45.118345 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cd29af49-90ea-4c8e-842a-c93ab1f64769 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:45.126789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 830/1671] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:45.163194 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29af49-90ea-4c8e-842a-c93ab1f64769 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:45.165171 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29af49-90ea-4c8e-842a-c93ab1f64769 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:45.229437 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 842/1672] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:45 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:45.428921 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda446cf-bec3-42c7-add9-6e4af1c04620 service nova] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:45.430724 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dda446cf-bec3-42c7-add9-6e4af1c04620 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:45.834416 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda446cf-bec3-42c7-add9-6e4af1c04620 service nova] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:45.844325 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65295e6f-fc32-4f8a-941a-3b06da6f4851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:45.845056 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65295e6f-fc32-4f8a-941a-3b06da6f4851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:45.845341 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65295e6f-fc32-4f8a-941a-3b06da6f4851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:45.845619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65295e6f-fc32-4f8a-941a-3b06da6f4851 admin admin] PATH_INFO: `/v3/projects/a586b9a96c934f9db30983d83142208f` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:45.846891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65295e6f-fc32-4f8a-941a-3b06da6f4851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a586b9a96c934f9db30983d83142208f)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:45.848950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65295e6f-fc32-4f8a-941a-3b06da6f4851 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:46.033507 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-65295e6f-fc32-4f8a-941a-3b06da6f4851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a586b9a96c934f9db30983d83142208f', 'request_id': 'req-65295e6f-fc32-4f8a-941a-3b06da6f4851'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:46.055355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-65295e6f-fc32-4f8a-941a-3b06da6f4851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a586b9a96c934f9db30983d83142208f', 'request_id': 'req-65295e6f-fc32-4f8a-941a-3b06da6f4851'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:46.079293 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-65295e6f-fc32-4f8a-941a-3b06da6f4851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a586b9a96c934f9db30983d83142208f', 'request_id': 'req-65295e6f-fc32-4f8a-941a-3b06da6f4851'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:46.126806 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-65295e6f-fc32-4f8a-941a-3b06da6f4851 admin admin] The token cache is being invalidate because project a586b9a96c934f9db30983d83142208f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:55:46.127314 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-65295e6f-fc32-4f8a-941a-3b06da6f4851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:46.133758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-65295e6f-fc32-4f8a-941a-3b06da6f4851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:46.136126 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 831/1673] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:55:45 2022] DELETE /identity/v3/projects/a586b9a96c934f9db30983d83142208f => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:46.141441 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b00a9c-0238-4c83-b902-f27d74ebe060 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:46.142019 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b00a9c-0238-4c83-b902-f27d74ebe060 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:46.142219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b00a9c-0238-4c83-b902-f27d74ebe060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:46.200186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f3b00a9c-0238-4c83-b902-f27d74ebe060 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:46.201456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f3b00a9c-0238-4c83-b902-f27d74ebe060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:46.343305 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 843/1674] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:46.350500 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6202ea8-63e4-4de9-8bc2-47e795adb938 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:46.350788 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6202ea8-63e4-4de9-8bc2-47e795adb938 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:46.351065 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6202ea8-63e4-4de9-8bc2-47e795adb938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:46.404714 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d6202ea8-63e4-4de9-8bc2-47e795adb938 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:46.407982 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d6202ea8-63e4-4de9-8bc2-47e795adb938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:46.425836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 832/1675] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:46.427114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b00a9c-0238-4c83-b902-f27d74ebe060 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:46.443616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b00a9c-0238-4c83-b902-f27d74ebe060 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:46.459002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fe7c4-a29a-48d6-9615-54d021510634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:46.460259 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fe7c4-a29a-48d6-9615-54d021510634 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:46.460259 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fe7c4-a29a-48d6-9615-54d021510634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:46.460768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fe7c4-a29a-48d6-9615-54d021510634 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:46.461940 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06fe7c4-a29a-48d6-9615-54d021510634 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:46.462315 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06fe7c4-a29a-48d6-9615-54d021510634 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:46.464046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06fe7c4-a29a-48d6-9615-54d021510634 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:46.513609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 844/1676] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:46.518445 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6202ea8-63e4-4de9-8bc2-47e795adb938 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:46.549085 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6202ea8-63e4-4de9-8bc2-47e795adb938 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:46.566686 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89877ff6-cc2e-4689-823d-09cce7df9c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:46.570561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89877ff6-cc2e-4689-823d-09cce7df9c88 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:46.571672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89877ff6-cc2e-4689-823d-09cce7df9c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:46.572279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89877ff6-cc2e-4689-823d-09cce7df9c88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:46.574275 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89877ff6-cc2e-4689-823d-09cce7df9c88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:46.589310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89877ff6-cc2e-4689-823d-09cce7df9c88 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:46.629621 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:46.645241 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 833/1677] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:46 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:46.647692 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fe7c4-a29a-48d6-9615-54d021510634 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:46.663168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fe7c4-a29a-48d6-9615-54d021510634 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:46.735284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:46.752716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1f0c8-56f8-4108-8f86-77ed90055020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:46.756331 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1f0c8-56f8-4108-8f86-77ed90055020 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:46.756686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1f0c8-56f8-4108-8f86-77ed90055020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:46.756966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1f0c8-56f8-4108-8f86-77ed90055020 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:46.758783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1f0c8-56f8-4108-8f86-77ed90055020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:46.765325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1f0c8-56f8-4108-8f86-77ed90055020 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:46.857071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 845/1678] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:46 2022] POST /identity/v3/users => generated 288 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:46.862859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89877ff6-cc2e-4689-823d-09cce7df9c88 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:46.882318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89877ff6-cc2e-4689-823d-09cce7df9c88 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:46.896027 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b29f73-690b-4d7e-8fa9-8620b191c139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:46.896975 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b29f73-690b-4d7e-8fa9-8620b191c139 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:46.897429 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b29f73-690b-4d7e-8fa9-8620b191c139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:46.897944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b29f73-690b-4d7e-8fa9-8620b191c139 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:46.898755 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b29f73-690b-4d7e-8fa9-8620b191c139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:46.901859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b29f73-690b-4d7e-8fa9-8620b191c139 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:46.934304 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1f0c8-56f8-4108-8f86-77ed90055020 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:46.940895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 834/1679] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:46.948211 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1f0c8-56f8-4108-8f86-77ed90055020 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:46.958193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c6deb-1347-43c6-b084-0802712bff54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:46.958424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c6deb-1347-43c6-b084-0802712bff54 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:46.958709 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c6deb-1347-43c6-b084-0802712bff54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:46.958984 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c6deb-1347-43c6-b084-0802712bff54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:46.961253 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c6deb-1347-43c6-b084-0802712bff54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:46.969832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c6deb-1347-43c6-b084-0802712bff54 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:47.012969 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 846/1680] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:47.017449 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b29f73-690b-4d7e-8fa9-8620b191c139 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:47.030980 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b29f73-690b-4d7e-8fa9-8620b191c139 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:47.047732 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761169d0-deba-4cac-a9f3-1fa5afe7c063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:47.048541 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761169d0-deba-4cac-a9f3-1fa5afe7c063 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:47.048798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761169d0-deba-4cac-a9f3-1fa5afe7c063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:47.049083 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761169d0-deba-4cac-a9f3-1fa5afe7c063 admin admin] PATH_INFO: `/v3/projects/757709df42104f49916ed922429f2d80/users/3e6b47f2407b43b68230f9572c108846/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:47.081005 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761169d0-deba-4cac-a9f3-1fa5afe7c063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=757709df42104f49916ed922429f2d80, user_id=3e6b47f2407b43b68230f9572c108846, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:47.084801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761169d0-deba-4cac-a9f3-1fa5afe7c063 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:47.120731 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 835/1681] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:47 2022] PUT /identity/v3/projects/757709df42104f49916ed922429f2d80/users/3e6b47f2407b43b68230f9572c108846/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:47.133010 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c4209-e403-48d4-bf87-0601f542887c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:47.133314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c4209-e403-48d4-bf87-0601f542887c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:47.133579 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c4209-e403-48d4-bf87-0601f542887c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:47.216107 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-488c4209-e403-48d4-bf87-0601f542887c None None] MFA Rules not processed for user `3e6b47f2407b43b68230f9572c108846`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:47.217372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-488c4209-e403-48d4-bf87-0601f542887c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:47.332572 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761169d0-deba-4cac-a9f3-1fa5afe7c063 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:47.335618 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-761169d0-deba-4cac-a9f3-1fa5afe7c063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:47.519347 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 847/1682] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:47.526688 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c4209-e403-48d4-bf87-0601f542887c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:47.530422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-488c4209-e403-48d4-bf87-0601f542887c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:47.602011 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761169d0-deba-4cac-a9f3-1fa5afe7c063 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:47.614277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9a4ad-93a0-4a42-a697-f7cb872b3d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:47.614962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9a4ad-93a0-4a42-a697-f7cb872b3d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:47.616255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9a4ad-93a0-4a42-a697-f7cb872b3d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:47.616537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9a4ad-93a0-4a42-a697-f7cb872b3d3d admin admin] PATH_INFO: `/v3/projects/b56f14e1e9074c7185527a9f78b144b6` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:47.625692 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9a4ad-93a0-4a42-a697-f7cb872b3d3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b56f14e1e9074c7185527a9f78b144b6)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:47.633427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9a4ad-93a0-4a42-a697-f7cb872b3d3d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:47.701642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c4209-e403-48d4-bf87-0601f542887c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:47.728605 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-71d9a4ad-93a0-4a42-a697-f7cb872b3d3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b56f14e1e9074c7185527a9f78b144b6', 'request_id': 'req-71d9a4ad-93a0-4a42-a697-f7cb872b3d3d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.742845 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-71d9a4ad-93a0-4a42-a697-f7cb872b3d3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b56f14e1e9074c7185527a9f78b144b6', 'request_id': 'req-71d9a4ad-93a0-4a42-a697-f7cb872b3d3d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.755898 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-71d9a4ad-93a0-4a42-a697-f7cb872b3d3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b56f14e1e9074c7185527a9f78b144b6', 'request_id': 'req-71d9a4ad-93a0-4a42-a697-f7cb872b3d3d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.775540 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-71d9a4ad-93a0-4a42-a697-f7cb872b3d3d admin admin] The token cache is being invalidate because project b56f14e1e9074c7185527a9f78b144b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:55:47.776709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-71d9a4ad-93a0-4a42-a697-f7cb872b3d3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.778206 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-71d9a4ad-93a0-4a42-a697-f7cb872b3d3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:47.804822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 836/1683] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:55:47 2022] DELETE /identity/v3/projects/b56f14e1e9074c7185527a9f78b144b6 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:47.806638 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:47.807363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9a4ad-93a0-4a42-a697-f7cb872b3d3d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:47.811019 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-71d9a4ad-93a0-4a42-a697-f7cb872b3d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:47.829954 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7255ddf-32af-4062-b837-7e5fbae03f89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:47.831772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7255ddf-32af-4062-b837-7e5fbae03f89 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:47.832049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7255ddf-32af-4062-b837-7e5fbae03f89 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:47.832353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7255ddf-32af-4062-b837-7e5fbae03f89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:47.836949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a7255ddf-32af-4062-b837-7e5fbae03f89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:47.930873 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:47.959340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9a4ad-93a0-4a42-a697-f7cb872b3d3d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:47.972205 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1c587-a059-407d-8aa4-d55fe924ce81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:47.974474 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1c587-a059-407d-8aa4-d55fe924ce81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:47.976616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1c587-a059-407d-8aa4-d55fe924ce81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:47.977500 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1c587-a059-407d-8aa4-d55fe924ce81 admin admin] PATH_INFO: `/v3/users/de7603c42c5042a5a6d717a47ec5a65c` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:47.979672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1c587-a059-407d-8aa4-d55fe924ce81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de7603c42c5042a5a6d717a47ec5a65c)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:47.988884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1c587-a059-407d-8aa4-d55fe924ce81 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:47.990145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7255ddf-32af-4062-b837-7e5fbae03f89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:47.992588 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7255ddf-32af-4062-b837-7e5fbae03f89 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:48.010589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 848/1684] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:48.012848 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee180a-2498-4abd-923c-1a54a53b682c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:48.014331 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee180a-2498-4abd-923c-1a54a53b682c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:48.014936 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee180a-2498-4abd-923c-1a54a53b682c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:48.067016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7dee180a-2498-4abd-923c-1a54a53b682c None None] MFA Rules not processed for user `5c036cb4ba8e4a16bd9e884243e665c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:48.070210 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7dee180a-2498-4abd-923c-1a54a53b682c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:48.113269 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ddb1c587-a059-407d-8aa4-d55fe924ce81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de7603c42c5042a5a6d717a47ec5a65c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:48.136190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ddb1c587-a059-407d-8aa4-d55fe924ce81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de7603c42c5042a5a6d717a47ec5a65c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:48.161131 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ddb1c587-a059-407d-8aa4-d55fe924ce81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de7603c42c5042a5a6d717a47ec5a65c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:48.196975 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ddb1c587-a059-407d-8aa4-d55fe924ce81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de7603c42c5042a5a6d717a47ec5a65c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:48.199359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ddb1c587-a059-407d-8aa4-d55fe924ce81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de7603c42c5042a5a6d717a47ec5a65c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:48.213753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 837/1685] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:55:47 2022] DELETE /identity/v3/users/de7603c42c5042a5a6d717a47ec5a65c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:48.216508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2521e-8f0f-4b20-b489-e7cfa7178f30 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:48.230852 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2521e-8f0f-4b20-b489-e7cfa7178f30 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:48.231396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2521e-8f0f-4b20-b489-e7cfa7178f30 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:48.233686 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 838/1686] 10.222.0.67 () {54 vars in 912 bytes} [Wed Aug 24 08:55:48 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:55:48.252819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 849/1687] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:48.258788 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe01d7-c1f5-4ad0-a614-d08199c6a827 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:48.259430 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe01d7-c1f5-4ad0-a614-d08199c6a827 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:48.259921 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe01d7-c1f5-4ad0-a614-d08199c6a827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:48.271746 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cf3d9-2f9f-482e-ac7a-885256f6a303 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:48.272042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cf3d9-2f9f-482e-ac7a-885256f6a303 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:48.272354 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cf3d9-2f9f-482e-ac7a-885256f6a303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:48.311152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d28cf3d9-2f9f-482e-ac7a-885256f6a303 None None] MFA Rules not processed for user `5c036cb4ba8e4a16bd9e884243e665c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:48.314039 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d28cf3d9-2f9f-482e-ac7a-885256f6a303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:48.409526 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-5cbe01d7-c1f5-4ad0-a614-d08199c6a827 None None] MFA Rules not processed for user `2ee2e93901914b1d82607a3ae0e2acf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:48.412706 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5cbe01d7-c1f5-4ad0-a614-d08199c6a827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:48.527064 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 850/1688] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:48.529155 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cf3d9-2f9f-482e-ac7a-885256f6a303 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:48.532747 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d28cf3d9-2f9f-482e-ac7a-885256f6a303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:48.554917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 839/1689] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 24 08:55:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:48.555910 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe01d7-c1f5-4ad0-a614-d08199c6a827 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:48.557560 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5cbe01d7-c1f5-4ad0-a614-d08199c6a827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:48.617646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:48.677150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cf3d9-2f9f-482e-ac7a-885256f6a303 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:48.700417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fae403-23ba-4ec4-90cf-6bb9a6f50269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:48.701809 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fae403-23ba-4ec4-90cf-6bb9a6f50269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:48.703593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fae403-23ba-4ec4-90cf-6bb9a6f50269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:48.704008 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fae403-23ba-4ec4-90cf-6bb9a6f50269 admin admin] PATH_INFO: `/v3/users/3e6b47f2407b43b68230f9572c108846` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:48.706087 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fae403-23ba-4ec4-90cf-6bb9a6f50269 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e6b47f2407b43b68230f9572c108846)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:48.709092 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fae403-23ba-4ec4-90cf-6bb9a6f50269 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:48.757104 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe01d7-c1f5-4ad0-a614-d08199c6a827 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:48.783189 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ea152-5ac3-4b2a-a329-7e44a888d3fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:48.784044 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ea152-5ac3-4b2a-a329-7e44a888d3fb service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:48.784468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ea152-5ac3-4b2a-a329-7e44a888d3fb service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:48.784817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ea152-5ac3-4b2a-a329-7e44a888d3fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:48.786857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-448ea152-5ac3-4b2a-a329-7e44a888d3fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:48.799894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:48.812219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ea152-5ac3-4b2a-a329-7e44a888d3fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:48.819184 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ea152-5ac3-4b2a-a329-7e44a888d3fb service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:48.821865 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e5fae403-23ba-4ec4-90cf-6bb9a6f50269 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e6b47f2407b43b68230f9572c108846'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:48.823471 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e5fae403-23ba-4ec4-90cf-6bb9a6f50269 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e6b47f2407b43b68230f9572c108846'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:48.844395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e5fae403-23ba-4ec4-90cf-6bb9a6f50269 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e6b47f2407b43b68230f9572c108846'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:48.856048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 840/1690] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:48.877409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ea152-5ac3-4b2a-a329-7e44a888d3fb service nova] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:48.889701 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e5fae403-23ba-4ec4-90cf-6bb9a6f50269 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e6b47f2407b43b68230f9572c108846'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:48.892666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-448ea152-5ac3-4b2a-a329-7e44a888d3fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:48.921871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e5fae403-23ba-4ec4-90cf-6bb9a6f50269 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e6b47f2407b43b68230f9572c108846'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:48.930690 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 851/1691] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:55:48 2022] DELETE /identity/v3/users/3e6b47f2407b43b68230f9572c108846 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:48.936237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafe975-2156-4072-b317-2a5485af0d80 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:48.940472 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafe975-2156-4072-b317-2a5485af0d80 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:48.940913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafe975-2156-4072-b317-2a5485af0d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:48.954685 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ea152-5ac3-4b2a-a329-7e44a888d3fb service nova] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:49.042612 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4aafe975-2156-4072-b317-2a5485af0d80 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:49.044273 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4aafe975-2156-4072-b317-2a5485af0d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:49.185865 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 852/1692] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:49.197486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e77bc-f26c-4cb0-9c8e-60a57d758547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:49.198644 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e77bc-f26c-4cb0-9c8e-60a57d758547 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:49.199231 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e77bc-f26c-4cb0-9c8e-60a57d758547 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:49.199846 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e77bc-f26c-4cb0-9c8e-60a57d758547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:49.202887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4f2e77bc-f26c-4cb0-9c8e-60a57d758547 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:49.219611 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e77bc-f26c-4cb0-9c8e-60a57d758547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:49.221416 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e77bc-f26c-4cb0-9c8e-60a57d758547 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:49.252544 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafe975-2156-4072-b317-2a5485af0d80 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:49.254900 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4aafe975-2156-4072-b317-2a5485af0d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:49.265996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 841/1693] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:49.279204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafe975-2156-4072-b317-2a5485af0d80 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:49.285480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeba623-6b82-4a12-800c-606d6fa98df7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:49.286382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeba623-6b82-4a12-800c-606d6fa98df7 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:49.286722 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeba623-6b82-4a12-800c-606d6fa98df7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:49.286995 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeba623-6b82-4a12-800c-606d6fa98df7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:49.304255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeba623-6b82-4a12-800c-606d6fa98df7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:49.306047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeba623-6b82-4a12-800c-606d6fa98df7 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:49.328964 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 853/1694] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:49.752724 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883e66e-2475-4fa4-bdfd-7b6276bcd03e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:49.753151 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883e66e-2475-4fa4-bdfd-7b6276bcd03e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:49.754018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883e66e-2475-4fa4-bdfd-7b6276bcd03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:49.845994 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-6883e66e-2475-4fa4-bdfd-7b6276bcd03e None None] MFA Rules not processed for user `f0dc2dcc5327489c9ecc28a43ab7c0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:49.849892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6883e66e-2475-4fa4-bdfd-7b6276bcd03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:50.016378 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 842/1695] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:50.629056 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf054f-0229-4a35-88c6-1bce66de6e90 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:50.630215 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf054f-0229-4a35-88c6-1bce66de6e90 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:50.630695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf054f-0229-4a35-88c6-1bce66de6e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:50.668110 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c3cf054f-0229-4a35-88c6-1bce66de6e90 None None] MFA Rules not processed for user `f0d0eb9dc0f743c3985218901334b336`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:50.669996 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c3cf054f-0229-4a35-88c6-1bce66de6e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:50.738919 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:50.768711 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 854/1696] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:50.818560 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fe0bb-e0cf-4cf8-8cf8-e0a48e7e5d1e None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:50.819156 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fe0bb-e0cf-4cf8-8cf8-e0a48e7e5d1e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:50.819246 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fe0bb-e0cf-4cf8-8cf8-e0a48e7e5d1e None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:50.823054 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 843/1697] 10.222.0.67 () {54 vars in 912 bytes} [Wed Aug 24 08:55:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:55:50.830085 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547f0e2-c4ee-4fe1-8150-342215834c02 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:50.830460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547f0e2-c4ee-4fe1-8150-342215834c02 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:50.831685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547f0e2-c4ee-4fe1-8150-342215834c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:50.908743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7547f0e2-c4ee-4fe1-8150-342215834c02 None None] MFA Rules not processed for user `2ee2e93901914b1d82607a3ae0e2acf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:50.917587 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7547f0e2-c4ee-4fe1-8150-342215834c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:50.928068 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fe0bb-e0cf-4cf8-8cf8-e0a48e7e5d1e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:50.929973 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-481fe0bb-e0cf-4cf8-8cf8-e0a48e7e5d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:50.962073 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 855/1698] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 24 08:55:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:50.963461 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547f0e2-c4ee-4fe1-8150-342215834c02 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:50.965497 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7547f0e2-c4ee-4fe1-8150-342215834c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:50.989348 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547f0e2-c4ee-4fe1-8150-342215834c02 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:50.991021 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fe0bb-e0cf-4cf8-8cf8-e0a48e7e5d1e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:50.995083 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8002b-2f1b-4fd0-9b58-4c03c2aea412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:50.996306 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8002b-2f1b-4fd0-9b58-4c03c2aea412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:50.997251 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8002b-2f1b-4fd0-9b58-4c03c2aea412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:50.997750 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8002b-2f1b-4fd0-9b58-4c03c2aea412 admin admin] PATH_INFO: `/v3/users/f0c2b7f3f63f4b5395999af532060adf` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.002848 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8002b-2f1b-4fd0-9b58-4c03c2aea412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0c2b7f3f63f4b5395999af532060adf)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:51.005242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8002b-2f1b-4fd0-9b58-4c03c2aea412 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:51.011570 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb0ac1-2568-4572-bf19-9ad62c5faf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:51.012257 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb0ac1-2568-4572-bf19-9ad62c5faf37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.012540 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb0ac1-2568-4572-bf19-9ad62c5faf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.012817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb0ac1-2568-4572-bf19-9ad62c5faf37 admin admin] PATH_INFO: `/v3/projects/757709df42104f49916ed922429f2d80` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.016966 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb0ac1-2568-4572-bf19-9ad62c5faf37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=757709df42104f49916ed922429f2d80)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:51.023884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb0ac1-2568-4572-bf19-9ad62c5faf37 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:51.088503 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3cc8002b-2f1b-4fd0-9b58-4c03c2aea412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0c2b7f3f63f4b5395999af532060adf'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:51.089220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3cc8002b-2f1b-4fd0-9b58-4c03c2aea412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0c2b7f3f63f4b5395999af532060adf'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:51.100424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3cc8002b-2f1b-4fd0-9b58-4c03c2aea412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0c2b7f3f63f4b5395999af532060adf'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:51.112637 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3cc8002b-2f1b-4fd0-9b58-4c03c2aea412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0c2b7f3f63f4b5395999af532060adf'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:51.137291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3cc8002b-2f1b-4fd0-9b58-4c03c2aea412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0c2b7f3f63f4b5395999af532060adf'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:51.144977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 856/1699] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:55:50 2022] DELETE /identity/v3/users/f0c2b7f3f63f4b5395999af532060adf => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:51.145895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8002b-2f1b-4fd0-9b58-4c03c2aea412 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:51.148543 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3cc8002b-2f1b-4fd0-9b58-4c03c2aea412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:51.186894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-94eb0ac1-2568-4572-bf19-9ad62c5faf37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '757709df42104f49916ed922429f2d80', 'request_id': 'req-94eb0ac1-2568-4572-bf19-9ad62c5faf37'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:51.221094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-94eb0ac1-2568-4572-bf19-9ad62c5faf37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '757709df42104f49916ed922429f2d80', 'request_id': 'req-94eb0ac1-2568-4572-bf19-9ad62c5faf37'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:51.240346 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-94eb0ac1-2568-4572-bf19-9ad62c5faf37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '757709df42104f49916ed922429f2d80', 'request_id': 'req-94eb0ac1-2568-4572-bf19-9ad62c5faf37'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:51.251369 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8002b-2f1b-4fd0-9b58-4c03c2aea412 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:51.255668 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53342ea-c2ad-43d3-a62c-98f70dfb55fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:51.257210 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53342ea-c2ad-43d3-a62c-98f70dfb55fc service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.257661 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53342ea-c2ad-43d3-a62c-98f70dfb55fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.257942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53342ea-c2ad-43d3-a62c-98f70dfb55fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.260638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-94eb0ac1-2568-4572-bf19-9ad62c5faf37 admin admin] The token cache is being invalidate because project 757709df42104f49916ed922429f2d80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:55:51.260904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e53342ea-c2ad-43d3-a62c-98f70dfb55fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:51.261330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-94eb0ac1-2568-4572-bf19-9ad62c5faf37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:51.262500 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-94eb0ac1-2568-4572-bf19-9ad62c5faf37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:51.269513 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 844/1700] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:55:50 2022] DELETE /identity/v3/projects/757709df42104f49916ed922429f2d80 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:55:51.270713 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb0ac1-2568-4572-bf19-9ad62c5faf37 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:51.272736 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-94eb0ac1-2568-4572-bf19-9ad62c5faf37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:51.275194 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53342ea-c2ad-43d3-a62c-98f70dfb55fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:51.276797 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53342ea-c2ad-43d3-a62c-98f70dfb55fc service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:51.288780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 857/1701] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:51.290077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0686d0-5c51-4cae-aa54-b6f7c4949ade None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.290359 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0686d0-5c51-4cae-aa54-b6f7c4949ade None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.290632 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0686d0-5c51-4cae-aa54-b6f7c4949ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.315571 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8f0686d0-5c51-4cae-aa54-b6f7c4949ade None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:51.316844 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8f0686d0-5c51-4cae-aa54-b6f7c4949ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:51.318100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:51.381383 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:51.398758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb0ac1-2568-4572-bf19-9ad62c5faf37 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:51.403697 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 858/1702] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:51.405242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b38ec7-3a68-41c4-97b4-beac052697a8 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.405673 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b38ec7-3a68-41c4-97b4-beac052697a8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.406084 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b38ec7-3a68-41c4-97b4-beac052697a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.414231 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79698ef-0f1a-465e-afb4-238567dc9656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:51.415360 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79698ef-0f1a-465e-afb4-238567dc9656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.415774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79698ef-0f1a-465e-afb4-238567dc9656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.416179 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79698ef-0f1a-465e-afb4-238567dc9656 admin admin] PATH_INFO: `/v3/projects/9bda613af4384e8d841fe0f800aff5a1` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.419218 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79698ef-0f1a-465e-afb4-238567dc9656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bda613af4384e8d841fe0f800aff5a1)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:51.422673 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79698ef-0f1a-465e-afb4-238567dc9656 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:51.436003 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-42b38ec7-3a68-41c4-97b4-beac052697a8 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:51.437935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-42b38ec7-3a68-41c4-97b4-beac052697a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:51.446913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 859/1703] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:51.448696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a637710-8036-4206-ae95-32150f3cb20f None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.449195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a637710-8036-4206-ae95-32150f3cb20f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.449700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a637710-8036-4206-ae95-32150f3cb20f None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.451336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 860/1704] 10.222.0.67 () {54 vars in 961 bytes} [Wed Aug 24 08:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:55:51.452637 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a637710-8036-4206-ae95-32150f3cb20f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:51.454927 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9a637710-8036-4206-ae95-32150f3cb20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:51.470154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a637710-8036-4206-ae95-32150f3cb20f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:51.474275 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54827beb-b74c-4a4e-9fb9-dd4c8c683901 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:51.475439 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54827beb-b74c-4a4e-9fb9-dd4c8c683901 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.475439 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54827beb-b74c-4a4e-9fb9-dd4c8c683901 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.475633 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54827beb-b74c-4a4e-9fb9-dd4c8c683901 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.497405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54827beb-b74c-4a4e-9fb9-dd4c8c683901 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:51.499590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54827beb-b74c-4a4e-9fb9-dd4c8c683901 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:51.513757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 861/1705] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:51.529823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60335f3-df61-46b7-b01d-79ecc85cae04 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.530185 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60335f3-df61-46b7-b01d-79ecc85cae04 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.530484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60335f3-df61-46b7-b01d-79ecc85cae04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.533094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c79698ef-0f1a-465e-afb4-238567dc9656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bda613af4384e8d841fe0f800aff5a1', 'request_id': 'req-c79698ef-0f1a-465e-afb4-238567dc9656'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:51.543728 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c79698ef-0f1a-465e-afb4-238567dc9656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bda613af4384e8d841fe0f800aff5a1', 'request_id': 'req-c79698ef-0f1a-465e-afb4-238567dc9656'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:51.565174 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c79698ef-0f1a-465e-afb4-238567dc9656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bda613af4384e8d841fe0f800aff5a1', 'request_id': 'req-c79698ef-0f1a-465e-afb4-238567dc9656'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:51.580683 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b60335f3-df61-46b7-b01d-79ecc85cae04 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:51.581562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c79698ef-0f1a-465e-afb4-238567dc9656 admin admin] The token cache is being invalidate because project 9bda613af4384e8d841fe0f800aff5a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:55:51.581940 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b60335f3-df61-46b7-b01d-79ecc85cae04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:51.582493 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c79698ef-0f1a-465e-afb4-238567dc9656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:51.583914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c79698ef-0f1a-465e-afb4-238567dc9656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:51.591020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 845/1706] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:55:51 2022] DELETE /identity/v3/projects/9bda613af4384e8d841fe0f800aff5a1 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:51.592855 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846afec-a29e-4a1d-80b5-d5fcde5030cf None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.595576 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846afec-a29e-4a1d-80b5-d5fcde5030cf None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.596787 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846afec-a29e-4a1d-80b5-d5fcde5030cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.623029 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e846afec-a29e-4a1d-80b5-d5fcde5030cf None None] MFA Rules not processed for user `33a0844a1c8f4ea4932e12d53e5ee9ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:51.624769 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e846afec-a29e-4a1d-80b5-d5fcde5030cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:51.733165 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 846/1707] 10.222.0.67 () {58 vars in 1058 bytes} [Wed Aug 24 08:55:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:51.737374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c90d4-5557-4893-9a0e-e5a87c70c742 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.739967 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c90d4-5557-4893-9a0e-e5a87c70c742 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.740402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c90d4-5557-4893-9a0e-e5a87c70c742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.773920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e53c90d4-5557-4893-9a0e-e5a87c70c742 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:51.776216 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e53c90d4-5557-4893-9a0e-e5a87c70c742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:51.791934 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 862/1708] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:51.793913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60335f3-df61-46b7-b01d-79ecc85cae04 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:51.798424 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 847/1709] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:51.799218 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c90d4-5557-4893-9a0e-e5a87c70c742 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:51.819114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c90d4-5557-4893-9a0e-e5a87c70c742 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:51.820682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60335f3-df61-46b7-b01d-79ecc85cae04 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:51.827213 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa9a3d-7819-4087-8d3f-9a8d5fc88bb4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33a0844a1c8f4ea4932e12d53e5ee9ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:51.829360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa9a3d-7819-4087-8d3f-9a8d5fc88bb4 service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.829918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa9a3d-7819-4087-8d3f-9a8d5fc88bb4 service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.830893 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa9a3d-7819-4087-8d3f-9a8d5fc88bb4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.831212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd1109-4ff3-4f4f-b38c-c1e3af16a83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:51.832296 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd1109-4ff3-4f4f-b38c-c1e3af16a83f admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.834955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd1109-4ff3-4f4f-b38c-c1e3af16a83f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.835215 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd1109-4ff3-4f4f-b38c-c1e3af16a83f admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.836788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-abaa9a3d-7819-4087-8d3f-9a8d5fc88bb4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:51.838273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd1109-4ff3-4f4f-b38c-c1e3af16a83f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:51.838418 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd1109-4ff3-4f4f-b38c-c1e3af16a83f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:51.845878 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd1109-4ff3-4f4f-b38c-c1e3af16a83f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:51.869864 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 848/1710] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:51.870903 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa9a3d-7819-4087-8d3f-9a8d5fc88bb4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:51.877881 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa9a3d-7819-4087-8d3f-9a8d5fc88bb4 service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:51.879667 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93145ea1-95c7-4736-8a36-93f556490cb3 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.879961 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93145ea1-95c7-4736-8a36-93f556490cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.880269 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93145ea1-95c7-4736-8a36-93f556490cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.891668 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 863/1711] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:51.892965 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa9a3d-7819-4087-8d3f-9a8d5fc88bb4 service placement] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:51.918194 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa9a3d-7819-4087-8d3f-9a8d5fc88bb4 service placement] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:51.925894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17024e02-86da-4d97-9d26-c56527fd4513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:51.926577 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17024e02-86da-4d97-9d26-c56527fd4513 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:51.926933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17024e02-86da-4d97-9d26-c56527fd4513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:51.927315 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17024e02-86da-4d97-9d26-c56527fd4513 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:51.928032 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17024e02-86da-4d97-9d26-c56527fd4513 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:51.931773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17024e02-86da-4d97-9d26-c56527fd4513 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:51.945636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-93145ea1-95c7-4736-8a36-93f556490cb3 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:51.952784 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-93145ea1-95c7-4736-8a36-93f556490cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:51.974614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 849/1712] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:51.978881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93145ea1-95c7-4736-8a36-93f556490cb3 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:51.998485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93145ea1-95c7-4736-8a36-93f556490cb3 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:52.011856 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 864/1713] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:51 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:52.014420 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48b931-f5a4-4f62-a31a-6654f1e0e41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:52.015324 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48b931-f5a4-4f62-a31a-6654f1e0e41b admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:52.015796 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48b931-f5a4-4f62-a31a-6654f1e0e41b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:52.015994 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17024e02-86da-4d97-9d26-c56527fd4513 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:52.016629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48b931-f5a4-4f62-a31a-6654f1e0e41b admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:52.018268 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48b931-f5a4-4f62-a31a-6654f1e0e41b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:52.018790 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48b931-f5a4-4f62-a31a-6654f1e0e41b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:52.021735 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48b931-f5a4-4f62-a31a-6654f1e0e41b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:52.040792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 850/1714] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:55:52.045554 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48b931-f5a4-4f62-a31a-6654f1e0e41b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:52.070235 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48b931-f5a4-4f62-a31a-6654f1e0e41b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:52.118431 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:52.257098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8ac73-5f0c-4f2a-8cb2-cae9fc953e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:52.257765 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17024e02-86da-4d97-9d26-c56527fd4513 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:52.257878 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8ac73-5f0c-4f2a-8cb2-cae9fc953e8e admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:52.258363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8ac73-5f0c-4f2a-8cb2-cae9fc953e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:52.258632 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8ac73-5f0c-4f2a-8cb2-cae9fc953e8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:52.259461 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8ac73-5f0c-4f2a-8cb2-cae9fc953e8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:52.264583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8ac73-5f0c-4f2a-8cb2-cae9fc953e8e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:52.268655 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04b327-d003-406d-9abc-012ae37038d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:52.270871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04b327-d003-406d-9abc-012ae37038d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:52.271193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04b327-d003-406d-9abc-012ae37038d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:52.272216 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04b327-d003-406d-9abc-012ae37038d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:52.273985 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04b327-d003-406d-9abc-012ae37038d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:52.277774 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04b327-d003-406d-9abc-012ae37038d0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:52.323579 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 851/1715] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:52 2022] POST /identity/v3/projects => generated 383 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:52.324713 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8ac73-5f0c-4f2a-8cb2-cae9fc953e8e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:52.327083 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-50c8ac73-5f0c-4f2a-8cb2-cae9fc953e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:52.346439 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 865/1716] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:52 2022] POST /identity/v3/users => generated 291 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:52.350813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04b327-d003-406d-9abc-012ae37038d0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:52.376876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04b327-d003-406d-9abc-012ae37038d0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:52.424762 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8ac73-5f0c-4f2a-8cb2-cae9fc953e8e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:52.432672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7871b-1a99-4d14-a96a-ab3e974f5a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:52.433495 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7871b-1a99-4d14-a96a-ab3e974f5a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:52.433786 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7871b-1a99-4d14-a96a-ab3e974f5a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:52.434105 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7871b-1a99-4d14-a96a-ab3e974f5a33 admin admin] PATH_INFO: `/v3/projects/3daf165199e2404d98b908d32941e72a` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:52.435951 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7871b-1a99-4d14-a96a-ab3e974f5a33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3daf165199e2404d98b908d32941e72a)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:52.439600 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7871b-1a99-4d14-a96a-ab3e974f5a33 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:52.513964 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:52.531813 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-44b7871b-1a99-4d14-a96a-ab3e974f5a33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3daf165199e2404d98b908d32941e72a', 'request_id': 'req-44b7871b-1a99-4d14-a96a-ab3e974f5a33'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:52.552071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-44b7871b-1a99-4d14-a96a-ab3e974f5a33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3daf165199e2404d98b908d32941e72a', 'request_id': 'req-44b7871b-1a99-4d14-a96a-ab3e974f5a33'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:52.559327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-44b7871b-1a99-4d14-a96a-ab3e974f5a33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3daf165199e2404d98b908d32941e72a', 'request_id': 'req-44b7871b-1a99-4d14-a96a-ab3e974f5a33'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:52.572383 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-44b7871b-1a99-4d14-a96a-ab3e974f5a33 admin admin] The token cache is being invalidate because project 3daf165199e2404d98b908d32941e72a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:55:52.572727 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-44b7871b-1a99-4d14-a96a-ab3e974f5a33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:52.573464 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-44b7871b-1a99-4d14-a96a-ab3e974f5a33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:55:52.576545 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 852/1717] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:55:52 2022] DELETE /identity/v3/projects/3daf165199e2404d98b908d32941e72a => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:52.581191 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7871b-1a99-4d14-a96a-ab3e974f5a33 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:52.583700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f55797-e5fd-4af0-9b4c-adc8c52a4e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:52.584558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f55797-e5fd-4af0-9b4c-adc8c52a4e9e admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:52.584843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f55797-e5fd-4af0-9b4c-adc8c52a4e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:52.586188 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f55797-e5fd-4af0-9b4c-adc8c52a4e9e admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:52.587391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f55797-e5fd-4af0-9b4c-adc8c52a4e9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:52.589059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-44b7871b-1a99-4d14-a96a-ab3e974f5a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:52.591955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f55797-e5fd-4af0-9b4c-adc8c52a4e9e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:52.662133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 866/1718] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:52 2022] POST /identity/v3/users => generated 292 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:52.663609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591d0ee-20ea-43e7-a84a-5d99a5fbc01f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:52.666051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591d0ee-20ea-43e7-a84a-5d99a5fbc01f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:52.666336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591d0ee-20ea-43e7-a84a-5d99a5fbc01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:52.752286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:52.771400 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0591d0ee-20ea-43e7-a84a-5d99a5fbc01f None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:52.777406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0591d0ee-20ea-43e7-a84a-5d99a5fbc01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:52.865789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7871b-1a99-4d14-a96a-ab3e974f5a33 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:52.883849 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f060983-175a-4e23-b354-c197097ba320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:52.885014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f060983-175a-4e23-b354-c197097ba320 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:52.885628 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f060983-175a-4e23-b354-c197097ba320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:52.886232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f060983-175a-4e23-b354-c197097ba320 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:52.887240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f060983-175a-4e23-b354-c197097ba320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:52.890492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f060983-175a-4e23-b354-c197097ba320 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:52.927761 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 853/1719] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:52.928558 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f060983-175a-4e23-b354-c197097ba320 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:52.930651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6f060983-175a-4e23-b354-c197097ba320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:52.958927 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f060983-175a-4e23-b354-c197097ba320 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:52.969548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598d8c7-9e05-4aff-b026-2e6ecb7ae3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:52.970187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598d8c7-9e05-4aff-b026-2e6ecb7ae3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:52.970569 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598d8c7-9e05-4aff-b026-2e6ecb7ae3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:52.970874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598d8c7-9e05-4aff-b026-2e6ecb7ae3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:52.971542 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598d8c7-9e05-4aff-b026-2e6ecb7ae3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:52.971934 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 867/1720] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:52.973605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591d0ee-20ea-43e7-a84a-5d99a5fbc01f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:52.975763 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598d8c7-9e05-4aff-b026-2e6ecb7ae3f5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:52.987258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591d0ee-20ea-43e7-a84a-5d99a5fbc01f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:52.994060 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573cf36-6c5a-49e2-8a54-57fa6b93609d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:52.994816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573cf36-6c5a-49e2-8a54-57fa6b93609d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:52.995249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573cf36-6c5a-49e2-8a54-57fa6b93609d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:52.995618 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573cf36-6c5a-49e2-8a54-57fa6b93609d admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:52.996495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573cf36-6c5a-49e2-8a54-57fa6b93609d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:52.998216 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573cf36-6c5a-49e2-8a54-57fa6b93609d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:53.015129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 854/1721] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:53.015129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b268cb8-c55b-4fac-9c3b-fa5f2a81ef2c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:53.015129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b268cb8-c55b-4fac-9c3b-fa5f2a81ef2c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:53.015129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b268cb8-c55b-4fac-9c3b-fa5f2a81ef2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:53.015777 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 868/1722] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:53.016939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573cf36-6c5a-49e2-8a54-57fa6b93609d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:53.049374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573cf36-6c5a-49e2-8a54-57fa6b93609d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:53.058475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6726e715-a562-4d8a-93bf-e954d6cb8ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:53.059292 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726e715-a562-4d8a-93bf-e954d6cb8ec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:53.059716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726e715-a562-4d8a-93bf-e954d6cb8ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:53.060540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726e715-a562-4d8a-93bf-e954d6cb8ec1 admin admin] PATH_INFO: `/v3/projects/2978768e24234d7fabc46f44299bcaa5/users/941a17a9e94f4e87973f87c1bba4c8a8/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:53.088763 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6726e715-a562-4d8a-93bf-e954d6cb8ec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2978768e24234d7fabc46f44299bcaa5, user_id=941a17a9e94f4e87973f87c1bba4c8a8, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:53.093079 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6726e715-a562-4d8a-93bf-e954d6cb8ec1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:53.093283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1b268cb8-c55b-4fac-9c3b-fa5f2a81ef2c None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:53.097830 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1b268cb8-c55b-4fac-9c3b-fa5f2a81ef2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:53.121437 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 869/1723] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:53 2022] PUT /identity/v3/projects/2978768e24234d7fabc46f44299bcaa5/users/941a17a9e94f4e87973f87c1bba4c8a8/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:53.122254 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 855/1724] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:53.123001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6726e715-a562-4d8a-93bf-e954d6cb8ec1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:53.125877 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b268cb8-c55b-4fac-9c3b-fa5f2a81ef2c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:53.156767 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b268cb8-c55b-4fac-9c3b-fa5f2a81ef2c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:53.340044 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6726e715-a562-4d8a-93bf-e954d6cb8ec1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:53.364447 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f89c7-b61c-4231-a11e-4bd5a32762e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:53.365451 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f89c7-b61c-4231-a11e-4bd5a32762e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:53.365927 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f89c7-b61c-4231-a11e-4bd5a32762e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:53.366339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c232a-c622-4686-aa87-e6800e127bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:53.367781 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c232a-c622-4686-aa87-e6800e127bff admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:53.367980 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f89c7-b61c-4231-a11e-4bd5a32762e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:53.368494 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c232a-c622-4686-aa87-e6800e127bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:53.369728 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c232a-c622-4686-aa87-e6800e127bff admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:53.376141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f89c7-b61c-4231-a11e-4bd5a32762e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:53.377009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c232a-c622-4686-aa87-e6800e127bff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:55:53.378288 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c232a-c622-4686-aa87-e6800e127bff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:53.383141 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c232a-c622-4686-aa87-e6800e127bff admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:53.407274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f89c7-b61c-4231-a11e-4bd5a32762e6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:53.410229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 856/1725] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:55:53.412644 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b6dfc-b8e4-4d95-ad86-09073aeaab80 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:53.413592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b6dfc-b8e4-4d95-ad86-09073aeaab80 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:53.414339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b6dfc-b8e4-4d95-ad86-09073aeaab80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:53.449765 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 870/1726] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:53.449765 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f89c7-b61c-4231-a11e-4bd5a32762e6 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:53.472868 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-532b6dfc-b8e4-4d95-ad86-09073aeaab80 None None] MFA Rules not processed for user `941a17a9e94f4e87973f87c1bba4c8a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:53.473023 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f89c7-b61c-4231-a11e-4bd5a32762e6 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:53.479175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-532b6dfc-b8e4-4d95-ad86-09073aeaab80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:53.496577 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd0503-3059-4082-9750-0bdfbbac9c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:53.496905 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd0503-3059-4082-9750-0bdfbbac9c63 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:53.499188 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd0503-3059-4082-9750-0bdfbbac9c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:53.499498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd0503-3059-4082-9750-0bdfbbac9c63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:53.508624 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd0503-3059-4082-9750-0bdfbbac9c63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:53.528388 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd0503-3059-4082-9750-0bdfbbac9c63 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:53.619011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:53.657887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 871/1727] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:53 2022] POST /identity/v3/projects => generated 363 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:53.660839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd0503-3059-4082-9750-0bdfbbac9c63 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:53.727407 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 857/1728] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:53.735746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b6dfc-b8e4-4d95-ad86-09073aeaab80 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:53.818445 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b6dfc-b8e4-4d95-ad86-09073aeaab80 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:53.911062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:54.099976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd0503-3059-4082-9750-0bdfbbac9c63 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:54.110045 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee3ac1-f3d5-49b9-98bb-5f5fcb455269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:54.111068 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee3ac1-f3d5-49b9-98bb-5f5fcb455269 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:54.111566 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee3ac1-f3d5-49b9-98bb-5f5fcb455269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:54.112067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee3ac1-f3d5-49b9-98bb-5f5fcb455269 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:54.113349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee3ac1-f3d5-49b9-98bb-5f5fcb455269 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:54.130789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee3ac1-f3d5-49b9-98bb-5f5fcb455269 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:54.138565 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401600e-5b1c-4349-91a6-4e02fa94809e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:54.140323 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401600e-5b1c-4349-91a6-4e02fa94809e admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:54.143463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401600e-5b1c-4349-91a6-4e02fa94809e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:54.147067 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401600e-5b1c-4349-91a6-4e02fa94809e admin admin] PATH_INFO: `/v3/projects/93108f621fb0470c895c6b9634c882b2/users/0f5600cb83d240cfab8efb49f927a9f2/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:54.243837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401600e-5b1c-4349-91a6-4e02fa94809e admin admin] RBAC: Authorizing `identity:create_grant(project_id=93108f621fb0470c895c6b9634c882b2, user_id=0f5600cb83d240cfab8efb49f927a9f2, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:54.248217 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 858/1729] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:53 2022] POST /identity/v3/users => generated 282 bytes in 514 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:54.250318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee3ac1-f3d5-49b9-98bb-5f5fcb455269 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:54.253989 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401600e-5b1c-4349-91a6-4e02fa94809e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:54.340554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 872/1730] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:53 2022] PUT /identity/v3/projects/93108f621fb0470c895c6b9634c882b2/users/0f5600cb83d240cfab8efb49f927a9f2/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:54.341781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401600e-5b1c-4349-91a6-4e02fa94809e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:54.344816 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee3ac1-f3d5-49b9-98bb-5f5fcb455269 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:54.580668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e0d0f-25da-44a9-b9d8-453fa11e82cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:54.581739 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e0d0f-25da-44a9-b9d8-453fa11e82cc admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:54.582188 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e0d0f-25da-44a9-b9d8-453fa11e82cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:54.582602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e0d0f-25da-44a9-b9d8-453fa11e82cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:54.585370 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e0d0f-25da-44a9-b9d8-453fa11e82cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:54.588023 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e0d0f-25da-44a9-b9d8-453fa11e82cc admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:54.606486 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401600e-5b1c-4349-91a6-4e02fa94809e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:54.640307 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea52da-3aab-47af-beec-6134ac17ddc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:54.641247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea52da-3aab-47af-beec-6134ac17ddc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:54.641677 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea52da-3aab-47af-beec-6134ac17ddc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:54.642726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea52da-3aab-47af-beec-6134ac17ddc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:54.648057 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea52da-3aab-47af-beec-6134ac17ddc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:54.650106 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea52da-3aab-47af-beec-6134ac17ddc3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:54.672373 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 859/1731] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:54 2022] POST /identity/v3/projects => generated 381 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:54.679888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ff5be-75ac-4357-95bf-d8389aa91890 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:54.683386 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ff5be-75ac-4357-95bf-d8389aa91890 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:54.683866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ff5be-75ac-4357-95bf-d8389aa91890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:54.690300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 873/1732] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:54.695262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea52da-3aab-47af-beec-6134ac17ddc3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:54.731538 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea52da-3aab-47af-beec-6134ac17ddc3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:54.783891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-3e1ff5be-75ac-4357-95bf-d8389aa91890 None None] MFA Rules not processed for user `0f5600cb83d240cfab8efb49f927a9f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:54.785427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3e1ff5be-75ac-4357-95bf-d8389aa91890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:54.957574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef261e-93f2-4d88-8a9f-8f95028f5878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:54.960588 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef261e-93f2-4d88-8a9f-8f95028f5878 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:54.962279 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef261e-93f2-4d88-8a9f-8f95028f5878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:54.963488 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef261e-93f2-4d88-8a9f-8f95028f5878 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:54.966035 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef261e-93f2-4d88-8a9f-8f95028f5878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:54.974151 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 860/1733] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:54.974734 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef261e-93f2-4d88-8a9f-8f95028f5878 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:54.975786 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ff5be-75ac-4357-95bf-d8389aa91890 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:55.001997 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ff5be-75ac-4357-95bf-d8389aa91890 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:55.017597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923bf8c-09b5-4bca-aa78-bb8e9a996866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:55.018511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923bf8c-09b5-4bca-aa78-bb8e9a996866 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:55.019009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923bf8c-09b5-4bca-aa78-bb8e9a996866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:55.019437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923bf8c-09b5-4bca-aa78-bb8e9a996866 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:55.020318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923bf8c-09b5-4bca-aa78-bb8e9a996866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:55.022485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923bf8c-09b5-4bca-aa78-bb8e9a996866 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:55.060401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 861/1734] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:55.061272 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923bf8c-09b5-4bca-aa78-bb8e9a996866 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:55.094758 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 874/1735] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:54 2022] POST /identity/v3/users => generated 297 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:55.094758 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef261e-93f2-4d88-8a9f-8f95028f5878 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:55.121446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923bf8c-09b5-4bca-aa78-bb8e9a996866 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:55.140318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458afac3-e8b9-4dc4-b4b0-3c73b83ce5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:55.141148 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458afac3-e8b9-4dc4-b4b0-3c73b83ce5af admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:55.141429 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458afac3-e8b9-4dc4-b4b0-3c73b83ce5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:55.141707 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458afac3-e8b9-4dc4-b4b0-3c73b83ce5af admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:55.142408 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458afac3-e8b9-4dc4-b4b0-3c73b83ce5af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:55.155485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef261e-93f2-4d88-8a9f-8f95028f5878 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:55.156986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458afac3-e8b9-4dc4-b4b0-3c73b83ce5af admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:55.182527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1134a0-34d2-4aa4-98e7-bf67e7b8ab3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:55.185905 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1134a0-34d2-4aa4-98e7-bf67e7b8ab3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:55.187033 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1134a0-34d2-4aa4-98e7-bf67e7b8ab3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:55.190121 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1134a0-34d2-4aa4-98e7-bf67e7b8ab3e admin admin] PATH_INFO: `/v3/projects/2bcb2eb9d9e34e48958b91e8f849333e/users/dd5646656eec452fb34debe4d4c8b479/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:55.196223 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:55.212839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:55.213254 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 862/1736] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:55 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:55.214193 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458afac3-e8b9-4dc4-b4b0-3c73b83ce5af admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:55.255769 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1134a0-34d2-4aa4-98e7-bf67e7b8ab3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bcb2eb9d9e34e48958b91e8f849333e, user_id=dd5646656eec452fb34debe4d4c8b479, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:55.258781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1134a0-34d2-4aa4-98e7-bf67e7b8ab3e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:55.299067 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 875/1737] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:55 2022] PUT /identity/v3/projects/2bcb2eb9d9e34e48958b91e8f849333e/users/dd5646656eec452fb34debe4d4c8b479/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:55.300756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd09f1-206b-412c-b776-de7cfbfcf445 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:55.302307 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd09f1-206b-412c-b776-de7cfbfcf445 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:55.302606 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd09f1-206b-412c-b776-de7cfbfcf445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:55.349477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-22cd09f1-206b-412c-b776-de7cfbfcf445 None None] MFA Rules not processed for user `8ee5364270094596b64b08a8ac56f6ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:55.351750 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-22cd09f1-206b-412c-b776-de7cfbfcf445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:55.514698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458afac3-e8b9-4dc4-b4b0-3c73b83ce5af admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:55.569955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 876/1738] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:55.581390 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97eac5-5096-4a66-9b1f-d76b448b822a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:55.581857 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97eac5-5096-4a66-9b1f-d76b448b822a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:55.583222 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97eac5-5096-4a66-9b1f-d76b448b822a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:55.685450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8f97eac5-5096-4a66-9b1f-d76b448b822a None None] MFA Rules not processed for user `bd0929f371e4442eb4eb6e12a96bbfc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:55.694647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8f97eac5-5096-4a66-9b1f-d76b448b822a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:55.818537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac704d-cb32-4b39-88c0-0c04914d989c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:55.824035 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac704d-cb32-4b39-88c0-0c04914d989c admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:55.825927 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac704d-cb32-4b39-88c0-0c04914d989c admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:55.826084 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac704d-cb32-4b39-88c0-0c04914d989c admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:55.840853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac704d-cb32-4b39-88c0-0c04914d989c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:55.852602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac704d-cb32-4b39-88c0-0c04914d989c admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:55.905047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 863/1739] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 691 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:55.907931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac704d-cb32-4b39-88c0-0c04914d989c admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:55.939341 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac704d-cb32-4b39-88c0-0c04914d989c admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:55.961182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad52db-8fc0-4136-b4cc-be3d563abea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:55.961912 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad52db-8fc0-4136-b4cc-be3d563abea7 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:55.964059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad52db-8fc0-4136-b4cc-be3d563abea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:55.964413 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad52db-8fc0-4136-b4cc-be3d563abea7 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:55.965105 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad52db-8fc0-4136-b4cc-be3d563abea7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:55.967114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad52db-8fc0-4136-b4cc-be3d563abea7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:56.010919 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 877/1740] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:56.012783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd1509-a532-4f32-89c9-e4ddbb645d77 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:56.013071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd1509-a532-4f32-89c9-e4ddbb645d77 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:56.013342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd1509-a532-4f32-89c9-e4ddbb645d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:56.083646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-18dd1509-a532-4f32-89c9-e4ddbb645d77 None None] MFA Rules not processed for user `dd5646656eec452fb34debe4d4c8b479`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:56.087390 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-18dd1509-a532-4f32-89c9-e4ddbb645d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:56.095270 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 864/1741] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:55 2022] POST /identity/v3/users => generated 298 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:56.097045 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad52db-8fc0-4136-b4cc-be3d563abea7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:56.164515 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad52db-8fc0-4136-b4cc-be3d563abea7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:56.191457 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bedd6-1c77-405b-a4c2-d39d622fa044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:56.197296 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bedd6-1c77-405b-a4c2-d39d622fa044 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:56.197594 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bedd6-1c77-405b-a4c2-d39d622fa044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:56.197879 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bedd6-1c77-405b-a4c2-d39d622fa044 admin admin] PATH_INFO: `/v3/projects/8654e1a62bb24fa8904c22704dd5c7c2/users/7ff9400855574ca0878b7569536454c2/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:56.280379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8bedd6-1c77-405b-a4c2-d39d622fa044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8654e1a62bb24fa8904c22704dd5c7c2, user_id=7ff9400855574ca0878b7569536454c2, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:56.290851 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8bedd6-1c77-405b-a4c2-d39d622fa044 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:56.356935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 878/1742] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:56.359059 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd1509-a532-4f32-89c9-e4ddbb645d77 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:56.379995 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd1509-a532-4f32-89c9-e4ddbb645d77 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:56.381492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 865/1743] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:56 2022] PUT /identity/v3/projects/8654e1a62bb24fa8904c22704dd5c7c2/users/7ff9400855574ca0878b7569536454c2/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:56.382279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bedd6-1c77-405b-a4c2-d39d622fa044 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:56.386743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1b8bedd6-1c77-405b-a4c2-d39d622fa044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:56.397998 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:56.545638 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96396271-25f7-42c6-b665-b534819960f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:56.546433 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96396271-25f7-42c6-b665-b534819960f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:56.546924 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96396271-25f7-42c6-b665-b534819960f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:56.547534 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96396271-25f7-42c6-b665-b534819960f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:56.561304 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96396271-25f7-42c6-b665-b534819960f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:56.571942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96396271-25f7-42c6-b665-b534819960f1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:56.626352 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:56.653782 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 879/1744] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:56.659507 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96396271-25f7-42c6-b665-b534819960f1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:56.734628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96396271-25f7-42c6-b665-b534819960f1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:56.736414 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bedd6-1c77-405b-a4c2-d39d622fa044 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:56.758733 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d1b1a-21bd-4717-94fb-99356c73367e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:56.759449 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d1b1a-21bd-4717-94fb-99356c73367e service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:56.759732 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d1b1a-21bd-4717-94fb-99356c73367e service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:56.760007 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d1b1a-21bd-4717-94fb-99356c73367e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:56.773352 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec0641-1cab-4c1e-9a82-fe228c0524ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:56.777096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec0641-1cab-4c1e-9a82-fe228c0524ac admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:56.783957 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec0641-1cab-4c1e-9a82-fe228c0524ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:56.786106 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec0641-1cab-4c1e-9a82-fe228c0524ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:56.789489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec0641-1cab-4c1e-9a82-fe228c0524ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:56.809066 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec0641-1cab-4c1e-9a82-fe228c0524ac admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:56.869506 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d1b1a-21bd-4717-94fb-99356c73367e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:56.871110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d1b1a-21bd-4717-94fb-99356c73367e service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:56.885337 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 866/1745] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:56.887294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d1b1a-21bd-4717-94fb-99356c73367e service nova] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:56.904500 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d1b1a-21bd-4717-94fb-99356c73367e service nova] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:56.911634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689c997-db04-4c38-ba4d-569bea2b6adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:56.912337 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689c997-db04-4c38-ba4d-569bea2b6adb admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:56.913068 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689c997-db04-4c38-ba4d-569bea2b6adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:56.913385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689c997-db04-4c38-ba4d-569bea2b6adb admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:56.914110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689c997-db04-4c38-ba4d-569bea2b6adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:56.916252 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689c997-db04-4c38-ba4d-569bea2b6adb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:56.933911 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 867/1746] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:56.936138 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689c997-db04-4c38-ba4d-569bea2b6adb admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:56.949875 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689c997-db04-4c38-ba4d-569bea2b6adb admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:56.958479 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b95e67-99b2-4e70-bbd0-d4bcf87c2ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:56.960295 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b95e67-99b2-4e70-bbd0-d4bcf87c2ae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:56.960796 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b95e67-99b2-4e70-bbd0-d4bcf87c2ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:56.961219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b95e67-99b2-4e70-bbd0-d4bcf87c2ae0 admin admin] PATH_INFO: `/v3/projects/fe640bba68d3494f9a7c8e93059d985c/users/f2628043d8574d1988c31f6dc96dacca/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:57.002672 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 880/1747] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:55:56 2022] POST /identity/v3/projects => generated 365 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:57.004133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec0641-1cab-4c1e-9a82-fe228c0524ac admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:57.016633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b95e67-99b2-4e70-bbd0-d4bcf87c2ae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe640bba68d3494f9a7c8e93059d985c, user_id=f2628043d8574d1988c31f6dc96dacca, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:57.030026 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b95e67-99b2-4e70-bbd0-d4bcf87c2ae0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:57.068018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 868/1748] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:56 2022] PUT /identity/v3/projects/fe640bba68d3494f9a7c8e93059d985c/users/f2628043d8574d1988c31f6dc96dacca/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:57.069952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b95e67-99b2-4e70-bbd0-d4bcf87c2ae0 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:57.298404 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b95e67-99b2-4e70-bbd0-d4bcf87c2ae0 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:57.301255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec0641-1cab-4c1e-9a82-fe228c0524ac admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:57.314680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc921d-0a7f-45ff-9f69-20456a6b9e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:57.315628 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc921d-0a7f-45ff-9f69-20456a6b9e2f admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:57.315628 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc921d-0a7f-45ff-9f69-20456a6b9e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:57.315902 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc921d-0a7f-45ff-9f69-20456a6b9e2f admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:57.316630 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc921d-0a7f-45ff-9f69-20456a6b9e2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:57.318590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc921d-0a7f-45ff-9f69-20456a6b9e2f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:57.325896 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036cbfb-219e-4a92-bac8-6e423e018933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:57.326950 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036cbfb-219e-4a92-bac8-6e423e018933 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:57.327346 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036cbfb-219e-4a92-bac8-6e423e018933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:57.327728 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036cbfb-219e-4a92-bac8-6e423e018933 admin admin] PATH_INFO: `/v3/projects/8654e1a62bb24fa8904c22704dd5c7c2/users/7ff9400855574ca0878b7569536454c2/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:57.330453 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0036cbfb-219e-4a92-bac8-6e423e018933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8654e1a62bb24fa8904c22704dd5c7c2, user_id=7ff9400855574ca0878b7569536454c2, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:57.341392 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0036cbfb-219e-4a92-bac8-6e423e018933 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:57.363640 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 869/1749] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:55:57 2022] POST /identity/v3/users => generated 289 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:55:57.364841 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc921d-0a7f-45ff-9f69-20456a6b9e2f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:57.376751 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 881/1750] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:57 2022] PUT /identity/v3/projects/8654e1a62bb24fa8904c22704dd5c7c2/users/7ff9400855574ca0878b7569536454c2/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:57.378303 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036cbfb-219e-4a92-bac8-6e423e018933 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:57.382444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc921d-0a7f-45ff-9f69-20456a6b9e2f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:57.626472 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:57.662655 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb27517-b995-44be-ba79-bd57c937b1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:57.664708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb27517-b995-44be-ba79-bd57c937b1c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:57.666975 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb27517-b995-44be-ba79-bd57c937b1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:57.670433 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb27517-b995-44be-ba79-bd57c937b1c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:57.670931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb27517-b995-44be-ba79-bd57c937b1c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:57.679105 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036cbfb-219e-4a92-bac8-6e423e018933 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:57.679906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb27517-b995-44be-ba79-bd57c937b1c7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:57.704640 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44cef0-419c-4037-aa42-5da601780f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:57.707734 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44cef0-419c-4037-aa42-5da601780f81 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:57.709926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44cef0-419c-4037-aa42-5da601780f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:57.712710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44cef0-419c-4037-aa42-5da601780f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:57.717580 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44cef0-419c-4037-aa42-5da601780f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:57.720785 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44cef0-419c-4037-aa42-5da601780f81 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:57.724064 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 870/1751] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:57.734839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b578ab-dff5-4243-ac89-43ee8ebedb8d None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:57.736481 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b578ab-dff5-4243-ac89-43ee8ebedb8d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:57.742788 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b578ab-dff5-4243-ac89-43ee8ebedb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:57.810999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 882/1752] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:57.818841 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cadf5-3762-4663-823c-5f0d6307b4fb None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:57.819052 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cadf5-3762-4663-823c-5f0d6307b4fb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:57.823866 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cadf5-3762-4663-823c-5f0d6307b4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:57.996156 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:58.013989 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-306cadf5-3762-4663-823c-5f0d6307b4fb None None] MFA Rules not processed for user `7ff9400855574ca0878b7569536454c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:58.016905 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-306cadf5-3762-4663-823c-5f0d6307b4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:58.084682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-10b578ab-dff5-4243-ac89-43ee8ebedb8d None None] MFA Rules not processed for user `b11abed8d4424a1e9957c3dc9aefadd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:58.097869 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-10b578ab-dff5-4243-ac89-43ee8ebedb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:58.354388 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 883/1753] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:57 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 543 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:58.356023 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306cadf5-3762-4663-823c-5f0d6307b4fb None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:58.379861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 871/1754] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 24 08:55:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 650 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:58.381338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b578ab-dff5-4243-ac89-43ee8ebedb8d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:58.418405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306cadf5-3762-4663-823c-5f0d6307b4fb None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:58.447521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b578ab-dff5-4243-ac89-43ee8ebedb8d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:58.453878 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c2e56-7dd6-4d2e-80d0-db968345d25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:58.456082 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c2e56-7dd6-4d2e-80d0-db968345d25a admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:58.456612 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c2e56-7dd6-4d2e-80d0-db968345d25a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:58.457825 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c2e56-7dd6-4d2e-80d0-db968345d25a admin admin] PATH_INFO: `/v3/projects/fe640bba68d3494f9a7c8e93059d985c/users/f2628043d8574d1988c31f6dc96dacca/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:58.468072 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c2e56-7dd6-4d2e-80d0-db968345d25a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe640bba68d3494f9a7c8e93059d985c, user_id=f2628043d8574d1988c31f6dc96dacca, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:58.479712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c2e56-7dd6-4d2e-80d0-db968345d25a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:58.486996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe3d44-564c-4bb0-8080-96c5186b2a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:58.488154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe3d44-564c-4bb0-8080-96c5186b2a3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:58.488646 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe3d44-564c-4bb0-8080-96c5186b2a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:58.489116 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe3d44-564c-4bb0-8080-96c5186b2a3a admin admin] PATH_INFO: `/v3/projects/6ecdb72526964b2588021b11a8b03c17/users/46de9e7e4eeb4a0f994509ca882e0cac/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:58.563995 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 884/1755] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:58 2022] PUT /identity/v3/projects/fe640bba68d3494f9a7c8e93059d985c/users/f2628043d8574d1988c31f6dc96dacca/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:58.565526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c2e56-7dd6-4d2e-80d0-db968345d25a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:58.574799 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0d9c2e56-7dd6-4d2e-80d0-db968345d25a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:58.580429 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe3d44-564c-4bb0-8080-96c5186b2a3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ecdb72526964b2588021b11a8b03c17, user_id=46de9e7e4eeb4a0f994509ca882e0cac, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:58.586988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe3d44-564c-4bb0-8080-96c5186b2a3a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:58.631634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 872/1756] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:58 2022] PUT /identity/v3/projects/6ecdb72526964b2588021b11a8b03c17/users/46de9e7e4eeb4a0f994509ca882e0cac/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:58.646722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48721ab-09d3-47af-90cd-be78a91109e2 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:58.647019 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48721ab-09d3-47af-90cd-be78a91109e2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:58.647270 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48721ab-09d3-47af-90cd-be78a91109e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:58.799383 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e48721ab-09d3-47af-90cd-be78a91109e2 None None] MFA Rules not processed for user `f2628043d8574d1988c31f6dc96dacca`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:55:58.800711 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e48721ab-09d3-47af-90cd-be78a91109e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:58.840976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c2e56-7dd6-4d2e-80d0-db968345d25a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:58.995647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1626cd-5785-4b15-904f-c0a0ac8d41e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:58.996710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1626cd-5785-4b15-904f-c0a0ac8d41e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:58.997175 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1626cd-5785-4b15-904f-c0a0ac8d41e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:58.997601 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1626cd-5785-4b15-904f-c0a0ac8d41e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.089779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 873/1757] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:55:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:55:59.091121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48721ab-09d3-47af-90cd-be78a91109e2 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:59.171172 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:59.252658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:55:59.285444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1626cd-5785-4b15-904f-c0a0ac8d41e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:59.288464 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1626cd-5785-4b15-904f-c0a0ac8d41e4 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:59.332872 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 885/1758] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 767 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:59.335475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48721ab-09d3-47af-90cd-be78a91109e2 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:59.336071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1626cd-5785-4b15-904f-c0a0ac8d41e4 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:59.338533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fc1626cd-5785-4b15-904f-c0a0ac8d41e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:59.348306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd60dd7-74e3-4dbe-9c53-db39a8543485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:59.351011 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd60dd7-74e3-4dbe-9c53-db39a8543485 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.352006 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd60dd7-74e3-4dbe-9c53-db39a8543485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.353813 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd60dd7-74e3-4dbe-9c53-db39a8543485 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.359285 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd60dd7-74e3-4dbe-9c53-db39a8543485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:59.366916 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd60dd7-74e3-4dbe-9c53-db39a8543485 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:59.405976 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 874/1759] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:55:59.405976 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd60dd7-74e3-4dbe-9c53-db39a8543485 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:59.418532 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0fd60dd7-74e3-4dbe-9c53-db39a8543485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:55:59.458766 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd60dd7-74e3-4dbe-9c53-db39a8543485 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:59.502487 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1952e78-56c6-463c-bca9-25f15e72db3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:59.508230 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1952e78-56c6-463c-bca9-25f15e72db3b service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.508581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1952e78-56c6-463c-bca9-25f15e72db3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.515558 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1952e78-56c6-463c-bca9-25f15e72db3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.581053 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1952e78-56c6-463c-bca9-25f15e72db3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:59.589969 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1952e78-56c6-463c-bca9-25f15e72db3b service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:59.679036 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 875/1760] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:59 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:55:59.681437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1952e78-56c6-463c-bca9-25f15e72db3b service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:55:59.764222 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1952e78-56c6-463c-bca9-25f15e72db3b service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:59.795334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbdbaee-0671-4c17-9bab-8167eb96c110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:55:59.799655 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdbaee-0671-4c17-9bab-8167eb96c110 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:55:59.802219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdbaee-0671-4c17-9bab-8167eb96c110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:55:59.804212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdbaee-0671-4c17-9bab-8167eb96c110 admin admin] PATH_INFO: `/v3/projects/6ecdb72526964b2588021b11a8b03c17/users/46de9e7e4eeb4a0f994509ca882e0cac/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:55:59.817965 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbdbaee-0671-4c17-9bab-8167eb96c110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ecdb72526964b2588021b11a8b03c17, user_id=46de9e7e4eeb4a0f994509ca882e0cac, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:55:59.821132 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbdbaee-0671-4c17-9bab-8167eb96c110 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:55:59.854039 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1626cd-5785-4b15-904f-c0a0ac8d41e4 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:55:59.880251 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 876/1761] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:55:59 2022] PUT /identity/v3/projects/6ecdb72526964b2588021b11a8b03c17/users/46de9e7e4eeb4a0f994509ca882e0cac/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:55:59.882757 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbdbaee-0671-4c17-9bab-8167eb96c110 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:00.111500 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbdbaee-0671-4c17-9bab-8167eb96c110 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:00.126856 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874312ca-8c7c-4058-a5cf-19064fc7e9ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:00.126856 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874312ca-8c7c-4058-a5cf-19064fc7e9ec service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:00.126856 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874312ca-8c7c-4058-a5cf-19064fc7e9ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:00.126856 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874312ca-8c7c-4058-a5cf-19064fc7e9ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:00.163504 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1dc96-2c6c-478a-95d3-1eace7a46347 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46b1417d4ac24c8982b2052f4e99042a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:00.164168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1dc96-2c6c-478a-95d3-1eace7a46347 service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:00.164454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1dc96-2c6c-478a-95d3-1eace7a46347 service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:00.164722 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1dc96-2c6c-478a-95d3-1eace7a46347 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:00.371204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874312ca-8c7c-4058-a5cf-19064fc7e9ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:00.381647 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874312ca-8c7c-4058-a5cf-19064fc7e9ec service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:00.396904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a1dc96-2c6c-478a-95d3-1eace7a46347 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:00.398674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a1dc96-2c6c-478a-95d3-1eace7a46347 service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:00.419141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 886/1762] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 24 08:55:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 1086 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:56:00.425042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75131d-2b74-45c1-8e83-0adee5e5e535 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:00.425422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75131d-2b74-45c1-8e83-0adee5e5e535 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:00.425983 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75131d-2b74-45c1-8e83-0adee5e5e535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:00.490817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 877/1763] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:55:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:00.499950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874312ca-8c7c-4058-a5cf-19064fc7e9ec service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:00.556236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874312ca-8c7c-4058-a5cf-19064fc7e9ec service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:00.556568 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4a75131d-2b74-45c1-8e83-0adee5e5e535 None None] MFA Rules not processed for user `46de9e7e4eeb4a0f994509ca882e0cac`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:00.557858 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4a75131d-2b74-45c1-8e83-0adee5e5e535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:00.577360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:00.602637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252cf53-6f88-43bf-8ad0-5b36c911f194 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:00.603391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252cf53-6f88-43bf-8ad0-5b36c911f194 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:00.607905 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252cf53-6f88-43bf-8ad0-5b36c911f194 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:00.608600 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252cf53-6f88-43bf-8ad0-5b36c911f194 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:00.928059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:01.011706 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252cf53-6f88-43bf-8ad0-5b36c911f194 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.020962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252cf53-6f88-43bf-8ad0-5b36c911f194 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.074036 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 887/1764] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 654 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:01.135361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 878/1765] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:01.286137 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75131d-2b74-45c1-8e83-0adee5e5e535 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:01.327537 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75131d-2b74-45c1-8e83-0adee5e5e535 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:01.391944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d205d76-f16f-4bb3-8c56-1ecf7d0fc853 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:01.392495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d205d76-f16f-4bb3-8c56-1ecf7d0fc853 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:01.416514 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d205d76-f16f-4bb3-8c56-1ecf7d0fc853 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:01.416514 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d205d76-f16f-4bb3-8c56-1ecf7d0fc853 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:01.534057 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d205d76-f16f-4bb3-8c56-1ecf7d0fc853 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:01.535938 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d205d76-f16f-4bb3-8c56-1ecf7d0fc853 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:01.624383 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 888/1766] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:01.898042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252cf53-6f88-43bf-8ad0-5b36c911f194 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:02.046932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d205d76-f16f-4bb3-8c56-1ecf7d0fc853 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:02.085906 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d205d76-f16f-4bb3-8c56-1ecf7d0fc853 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:02.121253 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3db9f-094a-48e5-b263-c75314d1fd20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:02.136435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3db9f-094a-48e5-b263-c75314d1fd20 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:02.138561 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3db9f-094a-48e5-b263-c75314d1fd20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:02.138841 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3db9f-094a-48e5-b263-c75314d1fd20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:02.329194 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252cf53-6f88-43bf-8ad0-5b36c911f194 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:02.375703 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929e8c0-f687-428d-837b-8fe181e4c7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:02.398559 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929e8c0-f687-428d-837b-8fe181e4c7cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:02.404024 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929e8c0-f687-428d-837b-8fe181e4c7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:02.405503 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929e8c0-f687-428d-837b-8fe181e4c7cd admin admin] PATH_INFO: `/v3/users/0f5600cb83d240cfab8efb49f927a9f2` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:02.425986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929e8c0-f687-428d-837b-8fe181e4c7cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f5600cb83d240cfab8efb49f927a9f2)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:02.436833 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929e8c0-f687-428d-837b-8fe181e4c7cd admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:02.672652 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a3db9f-094a-48e5-b263-c75314d1fd20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:02.692232 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a3db9f-094a-48e5-b263-c75314d1fd20 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:02.753980 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 889/1767] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 708 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:56:02.809178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4929e8c0-f687-428d-837b-8fe181e4c7cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f5600cb83d240cfab8efb49f927a9f2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:02.847511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4929e8c0-f687-428d-837b-8fe181e4c7cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f5600cb83d240cfab8efb49f927a9f2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:02.946878 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4929e8c0-f687-428d-837b-8fe181e4c7cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f5600cb83d240cfab8efb49f927a9f2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:03.001675 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4929e8c0-f687-428d-837b-8fe181e4c7cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f5600cb83d240cfab8efb49f927a9f2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:03.019899 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4929e8c0-f687-428d-837b-8fe181e4c7cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f5600cb83d240cfab8efb49f927a9f2'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:03.079745 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 879/1768] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:01 2022] DELETE /identity/v3/users/0f5600cb83d240cfab8efb49f927a9f2 => generated 0 bytes in 1182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:03.092771 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec438c-69af-4ffe-a38e-6c841926ee72 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:03.093885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec438c-69af-4ffe-a38e-6c841926ee72 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:03.094651 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec438c-69af-4ffe-a38e-6c841926ee72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:03.212373 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b3ec438c-69af-4ffe-a38e-6c841926ee72 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:03.220518 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b3ec438c-69af-4ffe-a38e-6c841926ee72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:03.713733 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 890/1769] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 623 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:04.995892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929e8c0-f687-428d-837b-8fe181e4c7cd admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:04.997629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4929e8c0-f687-428d-837b-8fe181e4c7cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:05.007417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1825a70-292c-47d8-9074-685ca8eb8324 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:05.008333 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1825a70-292c-47d8-9074-685ca8eb8324 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:05.008333 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1825a70-292c-47d8-9074-685ca8eb8324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:05.117279 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f1825a70-292c-47d8-9074-685ca8eb8324 None None] MFA Rules not processed for user `2ee2e93901914b1d82607a3ae0e2acf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:05.120007 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f1825a70-292c-47d8-9074-685ca8eb8324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:05.266218 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:05.452358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 891/1770] 10.222.0.50 () {58 vars in 1015 bytes} [Wed Aug 24 08:56:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:05.499274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1825a70-292c-47d8-9074-685ca8eb8324 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:05.501909 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f1825a70-292c-47d8-9074-685ca8eb8324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:05.540859 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1825a70-292c-47d8-9074-685ca8eb8324 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:05.553602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929e8c0-f687-428d-837b-8fe181e4c7cd admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:05.572955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566123f7-4511-4642-8927-b823be4d76a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:05.573617 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566123f7-4511-4642-8927-b823be4d76a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:05.576897 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566123f7-4511-4642-8927-b823be4d76a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:05.577179 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a585730-6ff3-485a-b904-590330455c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:05.577364 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566123f7-4511-4642-8927-b823be4d76a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:05.584046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a585730-6ff3-485a-b904-590330455c2f service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:05.588761 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a585730-6ff3-485a-b904-590330455c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:05.590007 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a585730-6ff3-485a-b904-590330455c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:05.607204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566123f7-4511-4642-8927-b823be4d76a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:05.611908 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566123f7-4511-4642-8927-b823be4d76a9 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:05.641178 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 892/1771] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:56:05.655027 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a585730-6ff3-485a-b904-590330455c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:05.674865 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a585730-6ff3-485a-b904-590330455c2f service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:05.777742 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566123f7-4511-4642-8927-b823be4d76a9 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:05.788252 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566123f7-4511-4642-8927-b823be4d76a9 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:05.794994 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 880/1772] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 800 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:56:05.811549 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c749354-2fbc-4f16-a8e7-39c16b7aa3bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:05.812227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c749354-2fbc-4f16-a8e7-39c16b7aa3bb service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:05.812567 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c749354-2fbc-4f16-a8e7-39c16b7aa3bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:05.812804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c749354-2fbc-4f16-a8e7-39c16b7aa3bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:05.867135 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c749354-2fbc-4f16-a8e7-39c16b7aa3bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:05.868922 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c749354-2fbc-4f16-a8e7-39c16b7aa3bb service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:05.923820 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 893/1773] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:08.249488 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a585730-6ff3-485a-b904-590330455c2f service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:08.253092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2a585730-6ff3-485a-b904-590330455c2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:08.286956 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a585730-6ff3-485a-b904-590330455c2f service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:08.293795 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7c5de-5bb6-4bf5-9556-80023e1857b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:08.294635 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7c5de-5bb6-4bf5-9556-80023e1857b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:08.295041 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7c5de-5bb6-4bf5-9556-80023e1857b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:08.295383 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7c5de-5bb6-4bf5-9556-80023e1857b3 admin admin] PATH_INFO: `/v3/users/dd5646656eec452fb34debe4d4c8b479` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:08.296934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e7c5de-5bb6-4bf5-9556-80023e1857b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd5646656eec452fb34debe4d4c8b479)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:08.299966 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e7c5de-5bb6-4bf5-9556-80023e1857b3 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:08.302602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:08.448411 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-76e7c5de-5bb6-4bf5-9556-80023e1857b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd5646656eec452fb34debe4d4c8b479'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:08.467506 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-76e7c5de-5bb6-4bf5-9556-80023e1857b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd5646656eec452fb34debe4d4c8b479'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:08.506708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-76e7c5de-5bb6-4bf5-9556-80023e1857b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd5646656eec452fb34debe4d4c8b479'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:08.529504 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-76e7c5de-5bb6-4bf5-9556-80023e1857b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd5646656eec452fb34debe4d4c8b479'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:08.532975 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-76e7c5de-5bb6-4bf5-9556-80023e1857b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd5646656eec452fb34debe4d4c8b479'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:08.552434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 881/1774] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:08 2022] DELETE /identity/v3/users/dd5646656eec452fb34debe4d4c8b479 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:08.556794 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fbdc8-3c15-4659-b5f9-67852813df6a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:08.557372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fbdc8-3c15-4659-b5f9-67852813df6a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:08.557372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fbdc8-3c15-4659-b5f9-67852813df6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:08.585590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-cb9fbdc8-3c15-4659-b5f9-67852813df6a None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:08.586837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cb9fbdc8-3c15-4659-b5f9-67852813df6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:08.753915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 894/1775] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:08.789906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7c5de-5bb6-4bf5-9556-80023e1857b3 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:08.791413 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-76e7c5de-5bb6-4bf5-9556-80023e1857b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:08.993426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7c5de-5bb6-4bf5-9556-80023e1857b3 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:09.015696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28b3ba-86a2-42ce-9f53-b229b72330f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:09.017079 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28b3ba-86a2-42ce-9f53-b229b72330f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:09.017407 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28b3ba-86a2-42ce-9f53-b229b72330f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:09.017667 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28b3ba-86a2-42ce-9f53-b229b72330f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:09.041165 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d28b3ba-86a2-42ce-9f53-b229b72330f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:09.042659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d28b3ba-86a2-42ce-9f53-b229b72330f4 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:09.076080 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 882/1776] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:09.413781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ff93e-23ed-47d4-a9c6-4819a9e5e8b3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:09.414079 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ff93e-23ed-47d4-a9c6-4819a9e5e8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:09.414490 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ff93e-23ed-47d4-a9c6-4819a9e5e8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:09.501375 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-562ff93e-23ed-47d4-a9c6-4819a9e5e8b3 None None] MFA Rules not processed for user `bd0929f371e4442eb4eb6e12a96bbfc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:09.510460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-562ff93e-23ed-47d4-a9c6-4819a9e5e8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:09.787414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 895/1777] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:10.021391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28b3ba-86a2-42ce-9f53-b229b72330f4 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:10.028070 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8d28b3ba-86a2-42ce-9f53-b229b72330f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:10.060148 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28b3ba-86a2-42ce-9f53-b229b72330f4 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:10.067571 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834bf30-ccb0-4ed7-85cd-0a60d7c6eb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:10.069756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834bf30-ccb0-4ed7-85cd-0a60d7c6eb3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:10.070322 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834bf30-ccb0-4ed7-85cd-0a60d7c6eb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:10.070967 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834bf30-ccb0-4ed7-85cd-0a60d7c6eb3d admin admin] PATH_INFO: `/v3/projects/93108f621fb0470c895c6b9634c882b2` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:10.072623 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834bf30-ccb0-4ed7-85cd-0a60d7c6eb3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=93108f621fb0470c895c6b9634c882b2)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:10.075147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834bf30-ccb0-4ed7-85cd-0a60d7c6eb3d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:10.204948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e834bf30-ccb0-4ed7-85cd-0a60d7c6eb3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93108f621fb0470c895c6b9634c882b2', 'request_id': 'req-e834bf30-ccb0-4ed7-85cd-0a60d7c6eb3d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:10.216375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e834bf30-ccb0-4ed7-85cd-0a60d7c6eb3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93108f621fb0470c895c6b9634c882b2', 'request_id': 'req-e834bf30-ccb0-4ed7-85cd-0a60d7c6eb3d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:10.227365 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e834bf30-ccb0-4ed7-85cd-0a60d7c6eb3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93108f621fb0470c895c6b9634c882b2', 'request_id': 'req-e834bf30-ccb0-4ed7-85cd-0a60d7c6eb3d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:10.244013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e834bf30-ccb0-4ed7-85cd-0a60d7c6eb3d admin admin] The token cache is being invalidate because project 93108f621fb0470c895c6b9634c882b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:10.244641 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e834bf30-ccb0-4ed7-85cd-0a60d7c6eb3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:10.245597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e834bf30-ccb0-4ed7-85cd-0a60d7c6eb3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:10.247721 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 883/1778] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:10 2022] DELETE /identity/v3/projects/93108f621fb0470c895c6b9634c882b2 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:10.252192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ff93e-23ed-47d4-a9c6-4819a9e5e8b3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:10.287327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ff93e-23ed-47d4-a9c6-4819a9e5e8b3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:10.490223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:10.500532 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce59b7-4df1-4f34-b2a1-a005a4fcb356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:10.502307 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce59b7-4df1-4f34-b2a1-a005a4fcb356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:10.502591 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce59b7-4df1-4f34-b2a1-a005a4fcb356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:10.502865 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce59b7-4df1-4f34-b2a1-a005a4fcb356 admin admin] PATH_INFO: `/v3/users/f2628043d8574d1988c31f6dc96dacca` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:10.504153 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce59b7-4df1-4f34-b2a1-a005a4fcb356 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2628043d8574d1988c31f6dc96dacca)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:10.507078 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce59b7-4df1-4f34-b2a1-a005a4fcb356 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:10.613812 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-40ce59b7-4df1-4f34-b2a1-a005a4fcb356 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2628043d8574d1988c31f6dc96dacca'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:10.616811 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-40ce59b7-4df1-4f34-b2a1-a005a4fcb356 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2628043d8574d1988c31f6dc96dacca'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:10.632418 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-40ce59b7-4df1-4f34-b2a1-a005a4fcb356 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2628043d8574d1988c31f6dc96dacca'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:10.647321 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-40ce59b7-4df1-4f34-b2a1-a005a4fcb356 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2628043d8574d1988c31f6dc96dacca'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:10.699292 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-40ce59b7-4df1-4f34-b2a1-a005a4fcb356 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2628043d8574d1988c31f6dc96dacca'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:10.715742 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 896/1779] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:10 2022] DELETE /identity/v3/users/f2628043d8574d1988c31f6dc96dacca => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:11.052460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3388b49-868d-4e0d-affb-5da2b8a590d1 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:11.053746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3388b49-868d-4e0d-affb-5da2b8a590d1 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:11.054422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3388b49-868d-4e0d-affb-5da2b8a590d1 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:11.057769 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 884/1780] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 24 08:56:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:56:11.105263 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd4679-bcd4-44a8-b718-befd60eae681 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:11.105820 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd4679-bcd4-44a8-b718-befd60eae681 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:11.106389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd4679-bcd4-44a8-b718-befd60eae681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:11.194149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-12bd4679-bcd4-44a8-b718-befd60eae681 None None] MFA Rules not processed for user `ca56dd7216fb4946946b2801e144fec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:11.195962 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-12bd4679-bcd4-44a8-b718-befd60eae681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:11.335635 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3388b49-868d-4e0d-affb-5da2b8a590d1 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:11.342993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 897/1781] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 24 08:56:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:11.345304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e3388b49-868d-4e0d-affb-5da2b8a590d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:11.393262 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:11.613560 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3388b49-868d-4e0d-affb-5da2b8a590d1 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:11.627261 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de45c9-9f8a-4616-9f74-863c07c8b930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:11.630334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de45c9-9f8a-4616-9f74-863c07c8b930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:11.630712 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de45c9-9f8a-4616-9f74-863c07c8b930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:11.631248 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de45c9-9f8a-4616-9f74-863c07c8b930 admin admin] PATH_INFO: `/v3/projects/2bcb2eb9d9e34e48958b91e8f849333e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:11.634113 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de45c9-9f8a-4616-9f74-863c07c8b930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bcb2eb9d9e34e48958b91e8f849333e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:11.638834 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de45c9-9f8a-4616-9f74-863c07c8b930 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:11.809093 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f9de45c9-9f8a-4616-9f74-863c07c8b930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bcb2eb9d9e34e48958b91e8f849333e', 'request_id': 'req-f9de45c9-9f8a-4616-9f74-863c07c8b930'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:11.826660 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f9de45c9-9f8a-4616-9f74-863c07c8b930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bcb2eb9d9e34e48958b91e8f849333e', 'request_id': 'req-f9de45c9-9f8a-4616-9f74-863c07c8b930'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:11.889291 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f9de45c9-9f8a-4616-9f74-863c07c8b930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bcb2eb9d9e34e48958b91e8f849333e', 'request_id': 'req-f9de45c9-9f8a-4616-9f74-863c07c8b930'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:11.944823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f9de45c9-9f8a-4616-9f74-863c07c8b930 admin admin] The token cache is being invalidate because project 2bcb2eb9d9e34e48958b91e8f849333e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:11.945698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f9de45c9-9f8a-4616-9f74-863c07c8b930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:11.966386 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f9de45c9-9f8a-4616-9f74-863c07c8b930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:11.975673 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 885/1782] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:11 2022] DELETE /identity/v3/projects/2bcb2eb9d9e34e48958b91e8f849333e => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:11.993917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd4679-bcd4-44a8-b718-befd60eae681 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:12.009271 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de45c9-9f8a-4616-9f74-863c07c8b930 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:12.016157 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f9de45c9-9f8a-4616-9f74-863c07c8b930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:12.041672 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd4679-bcd4-44a8-b718-befd60eae681 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:12.301775 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a3fa9-20dd-4e95-ad71-b867dbe7c3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:12.302583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a3fa9-20dd-4e95-ad71-b867dbe7c3a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:12.302918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a3fa9-20dd-4e95-ad71-b867dbe7c3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:12.303260 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a3fa9-20dd-4e95-ad71-b867dbe7c3a6 admin admin] PATH_INFO: `/v3/users/46de9e7e4eeb4a0f994509ca882e0cac` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:12.310171 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a3fa9-20dd-4e95-ad71-b867dbe7c3a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46de9e7e4eeb4a0f994509ca882e0cac)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:12.312247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a3fa9-20dd-4e95-ad71-b867dbe7c3a6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:12.316501 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de45c9-9f8a-4616-9f74-863c07c8b930 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:12.326319 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:12.326690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49becd-e13c-409b-986e-270abc44ad2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:12.327804 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49becd-e13c-409b-986e-270abc44ad2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:12.328176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49becd-e13c-409b-986e-270abc44ad2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:12.328523 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49becd-e13c-409b-986e-270abc44ad2b admin admin] PATH_INFO: `/v3/projects/fe640bba68d3494f9a7c8e93059d985c` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:12.331140 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49becd-e13c-409b-986e-270abc44ad2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe640bba68d3494f9a7c8e93059d985c)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:12.333155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49becd-e13c-409b-986e-270abc44ad2b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:12.449314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-345a3fa9-20dd-4e95-ad71-b867dbe7c3a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46de9e7e4eeb4a0f994509ca882e0cac'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:12.450412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-345a3fa9-20dd-4e95-ad71-b867dbe7c3a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46de9e7e4eeb4a0f994509ca882e0cac'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:12.452185 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:12.465497 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-345a3fa9-20dd-4e95-ad71-b867dbe7c3a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46de9e7e4eeb4a0f994509ca882e0cac'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:12.474750 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3b49becd-e13c-409b-986e-270abc44ad2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe640bba68d3494f9a7c8e93059d985c', 'request_id': 'req-3b49becd-e13c-409b-986e-270abc44ad2b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:12.488801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-345a3fa9-20dd-4e95-ad71-b867dbe7c3a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46de9e7e4eeb4a0f994509ca882e0cac'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:12.520719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-345a3fa9-20dd-4e95-ad71-b867dbe7c3a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46de9e7e4eeb4a0f994509ca882e0cac'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:12.521325 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3b49becd-e13c-409b-986e-270abc44ad2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe640bba68d3494f9a7c8e93059d985c', 'request_id': 'req-3b49becd-e13c-409b-986e-270abc44ad2b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:12.545630 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3b49becd-e13c-409b-986e-270abc44ad2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe640bba68d3494f9a7c8e93059d985c', 'request_id': 'req-3b49becd-e13c-409b-986e-270abc44ad2b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:12.552696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 898/1783] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:11 2022] DELETE /identity/v3/users/46de9e7e4eeb4a0f994509ca882e0cac => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:12.591959 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3b49becd-e13c-409b-986e-270abc44ad2b admin admin] The token cache is being invalidate because project fe640bba68d3494f9a7c8e93059d985c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:12.592854 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3b49becd-e13c-409b-986e-270abc44ad2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:12.594144 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3b49becd-e13c-409b-986e-270abc44ad2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:12.597782 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 886/1784] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:12 2022] DELETE /identity/v3/projects/fe640bba68d3494f9a7c8e93059d985c => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:12.606658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21d5b1-f7de-44c6-ae01-5fba77fc4fbd None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:12.609109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21d5b1-f7de-44c6-ae01-5fba77fc4fbd None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:12.609800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21d5b1-f7de-44c6-ae01-5fba77fc4fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:12.656124 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-fe21d5b1-f7de-44c6-ae01-5fba77fc4fbd None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:12.657890 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fe21d5b1-f7de-44c6-ae01-5fba77fc4fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:12.986232 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 899/1785] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:12.987491 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250891af-7663-4045-b2b1-7394782f6539 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:12.987819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250891af-7663-4045-b2b1-7394782f6539 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:12.988290 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250891af-7663-4045-b2b1-7394782f6539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:13.049464 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-250891af-7663-4045-b2b1-7394782f6539 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:13.050829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-250891af-7663-4045-b2b1-7394782f6539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:13.078263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 887/1786] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:13.082784 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21d5b1-f7de-44c6-ae01-5fba77fc4fbd None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:13.122488 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21d5b1-f7de-44c6-ae01-5fba77fc4fbd None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:13.133227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44eabc-673d-4850-8bad-1d72f59dbdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:13.134435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44eabc-673d-4850-8bad-1d72f59dbdc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:13.135756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44eabc-673d-4850-8bad-1d72f59dbdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:13.136318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44eabc-673d-4850-8bad-1d72f59dbdc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:13.137611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44eabc-673d-4850-8bad-1d72f59dbdc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:13.138129 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44eabc-673d-4850-8bad-1d72f59dbdc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:13.142367 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44eabc-673d-4850-8bad-1d72f59dbdc8 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:13.213490 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 900/1787] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:56:13.214442 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250891af-7663-4045-b2b1-7394782f6539 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:13.239866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250891af-7663-4045-b2b1-7394782f6539 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:13.257366 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d093359-95cd-499c-991c-ed1ace7f6f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:13.258012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d093359-95cd-499c-991c-ed1ace7f6f47 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:13.258429 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d093359-95cd-499c-991c-ed1ace7f6f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:13.258708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d093359-95cd-499c-991c-ed1ace7f6f47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:13.259625 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d093359-95cd-499c-991c-ed1ace7f6f47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:13.268334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d093359-95cd-499c-991c-ed1ace7f6f47 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:13.327678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44eabc-673d-4850-8bad-1d72f59dbdc8 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:13.330264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-da44eabc-673d-4850-8bad-1d72f59dbdc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:13.397919 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 888/1788] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:13 2022] POST /identity/v3/projects => generated 367 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:13.402272 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d093359-95cd-499c-991c-ed1ace7f6f47 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:13.585979 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d093359-95cd-499c-991c-ed1ace7f6f47 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:13.594159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83788b3c-407d-40bb-a262-2d1dceef16dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:13.595603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83788b3c-407d-40bb-a262-2d1dceef16dd admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:13.596017 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83788b3c-407d-40bb-a262-2d1dceef16dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:13.596482 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83788b3c-407d-40bb-a262-2d1dceef16dd admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:13.597290 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83788b3c-407d-40bb-a262-2d1dceef16dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:13.599513 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83788b3c-407d-40bb-a262-2d1dceef16dd admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:13.670135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 889/1789] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:13 2022] POST /identity/v3/users => generated 284 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:13.680747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83788b3c-407d-40bb-a262-2d1dceef16dd admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:13.737813 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83788b3c-407d-40bb-a262-2d1dceef16dd admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:13.739581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44eabc-673d-4850-8bad-1d72f59dbdc8 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:13.763972 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:13.797623 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026490f7-68be-435c-9c32-662158c2861a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:13.802179 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026490f7-68be-435c-9c32-662158c2861a admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:13.802379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026490f7-68be-435c-9c32-662158c2861a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:13.808878 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026490f7-68be-435c-9c32-662158c2861a admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:13.813583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026490f7-68be-435c-9c32-662158c2861a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:13.815798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026490f7-68be-435c-9c32-662158c2861a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:13.865726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 890/1790] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:13.867298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689da3b-91b7-4587-b34d-32a5124b1ae2 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:13.868820 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689da3b-91b7-4587-b34d-32a5124b1ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:13.869374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689da3b-91b7-4587-b34d-32a5124b1ae2 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:13.872130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 891/1791] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 24 08:56:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:56:13.875373 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689da3b-91b7-4587-b34d-32a5124b1ae2 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:13.926170 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01dd4e-4eb4-4f01-b1af-d8c0198a5df8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:13.926874 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01dd4e-4eb4-4f01-b1af-d8c0198a5df8 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:13.927168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01dd4e-4eb4-4f01-b1af-d8c0198a5df8 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:13.927508 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01dd4e-4eb4-4f01-b1af-d8c0198a5df8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:13.941438 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4b01dd4e-4eb4-4f01-b1af-d8c0198a5df8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:13.943702 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689da3b-91b7-4587-b34d-32a5124b1ae2 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:13.964306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7af308-b753-4262-9933-9e10afb59a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:13.964938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7af308-b753-4262-9933-9e10afb59a6e admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:13.965328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7af308-b753-4262-9933-9e10afb59a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:13.965708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7af308-b753-4262-9933-9e10afb59a6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:13.966478 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7af308-b753-4262-9933-9e10afb59a6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:13.968411 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7af308-b753-4262-9933-9e10afb59a6e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:13.984119 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b01dd4e-4eb4-4f01-b1af-d8c0198a5df8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:13.993605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b01dd4e-4eb4-4f01-b1af-d8c0198a5df8 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:14.008121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 892/1792] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:14.009929 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcab4be-138a-4b92-a5ee-9e9026ade57f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:14.014400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcab4be-138a-4b92-a5ee-9e9026ade57f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:14.019135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcab4be-138a-4b92-a5ee-9e9026ade57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:14.035860 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 901/1793] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 709 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:56:14.038164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01dd4e-4eb4-4f01-b1af-d8c0198a5df8 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:14.072488 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01dd4e-4eb4-4f01-b1af-d8c0198a5df8 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:14.095285 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-afcab4be-138a-4b92-a5ee-9e9026ade57f None None] MFA Rules not processed for user `ca56dd7216fb4946946b2801e144fec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:14.096814 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8862ebd-5cb0-4397-8c25-89f0bc748ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:14.098889 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8862ebd-5cb0-4397-8c25-89f0bc748ade admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:14.099476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8862ebd-5cb0-4397-8c25-89f0bc748ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:14.100120 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-afcab4be-138a-4b92-a5ee-9e9026ade57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:14.100949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8862ebd-5cb0-4397-8c25-89f0bc748ade admin admin] PATH_INFO: `/v3/projects/ac325a2b980843d2ba76fa295fe7293f/users/80ba72a7414241bf829be747dd6ab276/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:14.129052 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8862ebd-5cb0-4397-8c25-89f0bc748ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac325a2b980843d2ba76fa295fe7293f, user_id=80ba72a7414241bf829be747dd6ab276, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:14.134067 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8862ebd-5cb0-4397-8c25-89f0bc748ade admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:14.140492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 893/1794] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 24 08:56:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:14.179648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 902/1795] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:14 2022] PUT /identity/v3/projects/ac325a2b980843d2ba76fa295fe7293f/users/80ba72a7414241bf829be747dd6ab276/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:14.190858 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc3dc0-3dea-4d11-b860-0aa964c84bf2 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:14.191366 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc3dc0-3dea-4d11-b860-0aa964c84bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:14.191887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc3dc0-3dea-4d11-b860-0aa964c84bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:14.262792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-8bfc3dc0-3dea-4d11-b860-0aa964c84bf2 None None] MFA Rules not processed for user `80ba72a7414241bf829be747dd6ab276`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:14.276628 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8bfc3dc0-3dea-4d11-b860-0aa964c84bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:14.427052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:14.501833 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 894/1796] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:14.511083 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8862ebd-5cb0-4397-8c25-89f0bc748ade admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:14.513448 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a8862ebd-5cb0-4397-8c25-89f0bc748ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:14.732521 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8862ebd-5cb0-4397-8c25-89f0bc748ade admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:14.758330 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f0b88-e64e-4395-a0f0-a0884fe278e0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5979387f7aa54619b8d429368c306cc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:14.780633 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f0b88-e64e-4395-a0f0-a0884fe278e0 service glance] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:14.781223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f0b88-e64e-4395-a0f0-a0884fe278e0 service glance] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:14.781582 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f0b88-e64e-4395-a0f0-a0884fe278e0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:14.814295 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f0b88-e64e-4395-a0f0-a0884fe278e0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:14.825256 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f0b88-e64e-4395-a0f0-a0884fe278e0 service glance] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:14.858522 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 903/1797] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:15.380056 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc3dc0-3dea-4d11-b860-0aa964c84bf2 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:15.382311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8bfc3dc0-3dea-4d11-b860-0aa964c84bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:15.523487 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc3dc0-3dea-4d11-b860-0aa964c84bf2 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:15.532020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71198ed-e1be-4041-bff0-92c68f9ac2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:15.532824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71198ed-e1be-4041-bff0-92c68f9ac2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:15.533485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71198ed-e1be-4041-bff0-92c68f9ac2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:15.533838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71198ed-e1be-4041-bff0-92c68f9ac2fb admin admin] PATH_INFO: `/v3/projects/6ecdb72526964b2588021b11a8b03c17` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:15.535271 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71198ed-e1be-4041-bff0-92c68f9ac2fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ecdb72526964b2588021b11a8b03c17)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:15.537534 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71198ed-e1be-4041-bff0-92c68f9ac2fb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:15.661723 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f71198ed-e1be-4041-bff0-92c68f9ac2fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ecdb72526964b2588021b11a8b03c17', 'request_id': 'req-f71198ed-e1be-4041-bff0-92c68f9ac2fb'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:15.689856 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f71198ed-e1be-4041-bff0-92c68f9ac2fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ecdb72526964b2588021b11a8b03c17', 'request_id': 'req-f71198ed-e1be-4041-bff0-92c68f9ac2fb'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:15.710025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f71198ed-e1be-4041-bff0-92c68f9ac2fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ecdb72526964b2588021b11a8b03c17', 'request_id': 'req-f71198ed-e1be-4041-bff0-92c68f9ac2fb'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:15.751565 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f71198ed-e1be-4041-bff0-92c68f9ac2fb admin admin] The token cache is being invalidate because project 6ecdb72526964b2588021b11a8b03c17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:15.752002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f71198ed-e1be-4041-bff0-92c68f9ac2fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:15.752851 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f71198ed-e1be-4041-bff0-92c68f9ac2fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:15.760281 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 895/1798] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:15 2022] DELETE /identity/v3/projects/6ecdb72526964b2588021b11a8b03c17 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:15.761884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aca9a7-bde2-4343-a57e-0579449d7360 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:15.762770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aca9a7-bde2-4343-a57e-0579449d7360 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:15.763203 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aca9a7-bde2-4343-a57e-0579449d7360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:15.793243 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-23aca9a7-bde2-4343-a57e-0579449d7360 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:15.794578 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-23aca9a7-bde2-4343-a57e-0579449d7360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:15.995821 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:16.009548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71198ed-e1be-4041-bff0-92c68f9ac2fb admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:16.013088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f71198ed-e1be-4041-bff0-92c68f9ac2fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:16.074012 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 904/1799] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:16.085615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aca9a7-bde2-4343-a57e-0579449d7360 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:16.087268 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-23aca9a7-bde2-4343-a57e-0579449d7360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:16.117596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aca9a7-bde2-4343-a57e-0579449d7360 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:16.124808 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be06da-2626-4ec7-9454-229882f53d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:16.125717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be06da-2626-4ec7-9454-229882f53d87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:16.126614 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be06da-2626-4ec7-9454-229882f53d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:16.127149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be06da-2626-4ec7-9454-229882f53d87 admin admin] PATH_INFO: `/v3/users/941a17a9e94f4e87973f87c1bba4c8a8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:16.128650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be06da-2626-4ec7-9454-229882f53d87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=941a17a9e94f4e87973f87c1bba4c8a8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:16.130949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be06da-2626-4ec7-9454-229882f53d87 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:16.166430 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71198ed-e1be-4041-bff0-92c68f9ac2fb admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:16.179002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5df724-8dde-450c-a034-d2407f71d56a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:16.180945 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5df724-8dde-450c-a034-d2407f71d56a service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:16.181420 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5df724-8dde-450c-a034-d2407f71d56a service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:16.181893 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5df724-8dde-450c-a034-d2407f71d56a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:16.184793 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4d5df724-8dde-450c-a034-d2407f71d56a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:16.207905 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5df724-8dde-450c-a034-d2407f71d56a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:16.210968 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5df724-8dde-450c-a034-d2407f71d56a service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:16.238473 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 896/1800] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:16.242664 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a502d-0c13-4df1-9cf4-ca38ebcd1c96 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:16.243372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a502d-0c13-4df1-9cf4-ca38ebcd1c96 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:16.244467 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a502d-0c13-4df1-9cf4-ca38ebcd1c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:16.254000 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f1be06da-2626-4ec7-9454-229882f53d87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '941a17a9e94f4e87973f87c1bba4c8a8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:16.258896 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f1be06da-2626-4ec7-9454-229882f53d87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '941a17a9e94f4e87973f87c1bba4c8a8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:16.269218 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:16.270956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f1be06da-2626-4ec7-9454-229882f53d87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '941a17a9e94f4e87973f87c1bba4c8a8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:16.293121 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f1be06da-2626-4ec7-9454-229882f53d87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '941a17a9e94f4e87973f87c1bba4c8a8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:16.304962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-cc8a502d-0c13-4df1-9cf4-ca38ebcd1c96 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:16.311538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-cc8a502d-0c13-4df1-9cf4-ca38ebcd1c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:16.325204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f1be06da-2626-4ec7-9454-229882f53d87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '941a17a9e94f4e87973f87c1bba4c8a8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:16.336245 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 897/1801] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:16.339151 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a502d-0c13-4df1-9cf4-ca38ebcd1c96 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:16.343330 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 905/1802] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:16 2022] DELETE /identity/v3/users/941a17a9e94f4e87973f87c1bba4c8a8 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:16.345408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31f096-8fe5-4bf7-9560-905a7ae2f3fe None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:16.345756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31f096-8fe5-4bf7-9560-905a7ae2f3fe None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:16.345991 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31f096-8fe5-4bf7-9560-905a7ae2f3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:16.373415 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a502d-0c13-4df1-9cf4-ca38ebcd1c96 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:16.395743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1f31f096-8fe5-4bf7-9560-905a7ae2f3fe None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:16.397512 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1f31f096-8fe5-4bf7-9560-905a7ae2f3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:16.504157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 906/1803] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:16.572660 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48408ebc-a77c-4a15-b606-9f6730617e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:16.574753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48408ebc-a77c-4a15-b606-9f6730617e44 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:16.577760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48408ebc-a77c-4a15-b606-9f6730617e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:16.578250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48408ebc-a77c-4a15-b606-9f6730617e44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:16.581409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48408ebc-a77c-4a15-b606-9f6730617e44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:16.581946 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48408ebc-a77c-4a15-b606-9f6730617e44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:16.587527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48408ebc-a77c-4a15-b606-9f6730617e44 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:16.627200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 898/1804] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:56:16.635246 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31f096-8fe5-4bf7-9560-905a7ae2f3fe None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:16.636728 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48408ebc-a77c-4a15-b606-9f6730617e44 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:16.638856 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-48408ebc-a77c-4a15-b606-9f6730617e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:16.668700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31f096-8fe5-4bf7-9560-905a7ae2f3fe None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:16.686933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3904e8-0c6a-4fa2-8777-f81695d9df64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:16.687562 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3904e8-0c6a-4fa2-8777-f81695d9df64 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:16.687998 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3904e8-0c6a-4fa2-8777-f81695d9df64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:16.688196 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3904e8-0c6a-4fa2-8777-f81695d9df64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:16.689637 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3904e8-0c6a-4fa2-8777-f81695d9df64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:16.692384 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3904e8-0c6a-4fa2-8777-f81695d9df64 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:16.772639 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 907/1805] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:16 2022] POST /identity/v3/projects => generated 379 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:16.777720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3904e8-0c6a-4fa2-8777-f81695d9df64 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:16.819215 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48408ebc-a77c-4a15-b606-9f6730617e44 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:16.993668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3217013-e1b1-4d78-9b6d-822f095c9030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:16.994376 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3217013-e1b1-4d78-9b6d-822f095c9030 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:16.994664 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3217013-e1b1-4d78-9b6d-822f095c9030 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:16.995048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3217013-e1b1-4d78-9b6d-822f095c9030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:17.008822 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:17.071723 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3904e8-0c6a-4fa2-8777-f81695d9df64 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:17.084882 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156d35c-1576-407e-a88d-726eaebb412f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:17.086009 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156d35c-1576-407e-a88d-726eaebb412f admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:17.086554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156d35c-1576-407e-a88d-726eaebb412f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:17.087072 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156d35c-1576-407e-a88d-726eaebb412f admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:17.088602 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156d35c-1576-407e-a88d-726eaebb412f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:17.093603 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156d35c-1576-407e-a88d-726eaebb412f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:17.156996 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 908/1806] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:16 2022] POST /identity/v3/users => generated 290 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:17.160525 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3499d-9ea4-4eda-90a2-3434b8a6fa78 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:17.160525 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3499d-9ea4-4eda-90a2-3434b8a6fa78 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:17.161129 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3499d-9ea4-4eda-90a2-3434b8a6fa78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:17.196736 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:17.206153 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3217013-e1b1-4d78-9b6d-822f095c9030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:17.208014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3217013-e1b1-4d78-9b6d-822f095c9030 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:17.239143 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 899/1807] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:17.240446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3217013-e1b1-4d78-9b6d-822f095c9030 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:17.241535 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-76c3499d-9ea4-4eda-90a2-3434b8a6fa78 None None] MFA Rules not processed for user `b11abed8d4424a1e9957c3dc9aefadd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:17.242804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-76c3499d-9ea4-4eda-90a2-3434b8a6fa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:17.263467 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3217013-e1b1-4d78-9b6d-822f095c9030 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:17.273209 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb37a7-0f9b-4a8f-97a5-3dd519a9165a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:17.274250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb37a7-0f9b-4a8f-97a5-3dd519a9165a admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:17.274611 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb37a7-0f9b-4a8f-97a5-3dd519a9165a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:17.275026 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb37a7-0f9b-4a8f-97a5-3dd519a9165a admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:17.275783 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb37a7-0f9b-4a8f-97a5-3dd519a9165a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:17.278106 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb37a7-0f9b-4a8f-97a5-3dd519a9165a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:17.299765 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 900/1808] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:17.300947 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb37a7-0f9b-4a8f-97a5-3dd519a9165a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:17.317990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb37a7-0f9b-4a8f-97a5-3dd519a9165a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:17.325748 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a32d7-0e32-4d1b-8d90-96a72f34a4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:17.327632 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a32d7-0e32-4d1b-8d90-96a72f34a4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:17.328351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a32d7-0e32-4d1b-8d90-96a72f34a4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:17.330942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a32d7-0e32-4d1b-8d90-96a72f34a4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:17.332147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a32d7-0e32-4d1b-8d90-96a72f34a4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:17.336244 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a32d7-0e32-4d1b-8d90-96a72f34a4b5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:17.428521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 901/1809] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:17.431150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 909/1810] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 24 08:56:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:17.434328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a32d7-0e32-4d1b-8d90-96a72f34a4b5 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:17.458047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3499d-9ea4-4eda-90a2-3434b8a6fa78 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:17.460308 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-76c3499d-9ea4-4eda-90a2-3434b8a6fa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:17.498953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a32d7-0e32-4d1b-8d90-96a72f34a4b5 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:17.536117 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190307c-0661-4378-841f-8023264cf938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:17.537928 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190307c-0661-4378-841f-8023264cf938 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:17.538427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190307c-0661-4378-841f-8023264cf938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:17.538924 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190307c-0661-4378-841f-8023264cf938 admin admin] PATH_INFO: `/v3/projects/a711cf5e0dae48aba0509bd011be4261/users/b50eadbcc7fd4257b6c895edccd6fd1d/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:17.601550 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3190307c-0661-4378-841f-8023264cf938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a711cf5e0dae48aba0509bd011be4261, user_id=b50eadbcc7fd4257b6c895edccd6fd1d, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:17.607055 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3190307c-0661-4378-841f-8023264cf938 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:17.656703 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 902/1811] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:17 2022] PUT /identity/v3/projects/a711cf5e0dae48aba0509bd011be4261/users/b50eadbcc7fd4257b6c895edccd6fd1d/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:17.670103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df2f4b-1047-4c55-abc2-136adcdf2479 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:17.670592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df2f4b-1047-4c55-abc2-136adcdf2479 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:17.671053 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df2f4b-1047-4c55-abc2-136adcdf2479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:17.791923 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-46df2f4b-1047-4c55-abc2-136adcdf2479 None None] MFA Rules not processed for user `b50eadbcc7fd4257b6c895edccd6fd1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:17.794215 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-46df2f4b-1047-4c55-abc2-136adcdf2479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:17.818348 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3499d-9ea4-4eda-90a2-3434b8a6fa78 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:18.030755 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a31f47-c2e0-4ab6-8610-a978df0c14ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46b1417d4ac24c8982b2052f4e99042a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:18.034612 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a31f47-c2e0-4ab6-8610-a978df0c14ed service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:18.035718 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a31f47-c2e0-4ab6-8610-a978df0c14ed service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:18.036534 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a31f47-c2e0-4ab6-8610-a978df0c14ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:18.112187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 903/1812] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:18.116806 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df2f4b-1047-4c55-abc2-136adcdf2479 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:18.125981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:18.228599 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a31f47-c2e0-4ab6-8610-a978df0c14ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:18.232709 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a31f47-c2e0-4ab6-8610-a978df0c14ed service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:18.332040 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 910/1813] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 24 08:56:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 875 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 08:56:18.411081 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df2f4b-1047-4c55-abc2-136adcdf2479 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:18.456927 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c6374-38ea-4660-8506-4eb383d24165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:18.464583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c6374-38ea-4660-8506-4eb383d24165 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:18.465374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c6374-38ea-4660-8506-4eb383d24165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:18.466254 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c6374-38ea-4660-8506-4eb383d24165 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:18.474301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c6374-38ea-4660-8506-4eb383d24165 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:18.478340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c6374-38ea-4660-8506-4eb383d24165 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:18.587047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 904/1814] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:18 2022] POST /identity/v3/projects => generated 377 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:18.592504 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a31f47-c2e0-4ab6-8610-a978df0c14ed service swift] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:18.682866 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a31f47-c2e0-4ab6-8610-a978df0c14ed service swift] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:18.931650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29322215-bbf0-4589-9f5e-5913e4ac513f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:18.934169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29322215-bbf0-4589-9f5e-5913e4ac513f admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:18.937211 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29322215-bbf0-4589-9f5e-5913e4ac513f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:18.940613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29322215-bbf0-4589-9f5e-5913e4ac513f admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:18.945366 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29322215-bbf0-4589-9f5e-5913e4ac513f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:18.954978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29322215-bbf0-4589-9f5e-5913e4ac513f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:19.043530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 911/1815] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:18 2022] POST /identity/v3/users => generated 295 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:19.052231 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c6374-38ea-4660-8506-4eb383d24165 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:19.089608 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c6374-38ea-4660-8506-4eb383d24165 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:19.101010 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9afcb8-90db-49e9-822e-9f1ede036ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:19.102120 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9afcb8-90db-49e9-822e-9f1ede036ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:19.102587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9afcb8-90db-49e9-822e-9f1ede036ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:19.103196 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9afcb8-90db-49e9-822e-9f1ede036ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:19.105763 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9afcb8-90db-49e9-822e-9f1ede036ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:19.109194 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9afcb8-90db-49e9-822e-9f1ede036ab2 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:19.137784 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 905/1816] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:19.143698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29322215-bbf0-4589-9f5e-5913e4ac513f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:19.170944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29322215-bbf0-4589-9f5e-5913e4ac513f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:19.193162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be32ed-02aa-48be-88de-f4c80c190ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:19.194437 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be32ed-02aa-48be-88de-f4c80c190ea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:19.195043 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be32ed-02aa-48be-88de-f4c80c190ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:19.195866 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be32ed-02aa-48be-88de-f4c80c190ea1 admin admin] PATH_INFO: `/v3/projects/d8a6186b9f2d40c1a6a1e0f8b38f64a3/users/2890ef7cdb504a9bb6bc47d419a234f8/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:19.254785 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be32ed-02aa-48be-88de-f4c80c190ea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a6186b9f2d40c1a6a1e0f8b38f64a3, user_id=2890ef7cdb504a9bb6bc47d419a234f8, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:19.258476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be32ed-02aa-48be-88de-f4c80c190ea1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:19.308116 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 912/1817] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:19 2022] PUT /identity/v3/projects/d8a6186b9f2d40c1a6a1e0f8b38f64a3/users/2890ef7cdb504a9bb6bc47d419a234f8/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:19.308758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9afcb8-90db-49e9-822e-9f1ede036ab2 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:19.336972 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9afcb8-90db-49e9-822e-9f1ede036ab2 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:19.380142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:19.580406 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fcccb4-ff5f-4bf6-9b70-fb7b3ee5db0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:19.581273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcccb4-ff5f-4bf6-9b70-fb7b3ee5db0a admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:19.581551 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcccb4-ff5f-4bf6-9b70-fb7b3ee5db0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:19.581963 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcccb4-ff5f-4bf6-9b70-fb7b3ee5db0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:19.582469 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fcccb4-ff5f-4bf6-9b70-fb7b3ee5db0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:19.585742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fcccb4-ff5f-4bf6-9b70-fb7b3ee5db0a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:19.640558 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 906/1818] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:19.644355 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be32ed-02aa-48be-88de-f4c80c190ea1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:19.659254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be32ed-02aa-48be-88de-f4c80c190ea1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:19.665075 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243310a-621f-42b5-ab69-9d1f4f9023b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:19.665827 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243310a-621f-42b5-ab69-9d1f4f9023b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:19.666114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243310a-621f-42b5-ab69-9d1f4f9023b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:19.666498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243310a-621f-42b5-ab69-9d1f4f9023b6 admin admin] PATH_INFO: `/v3/projects/d8a6186b9f2d40c1a6a1e0f8b38f64a3/users/2890ef7cdb504a9bb6bc47d419a234f8/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:19.668734 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243310a-621f-42b5-ab69-9d1f4f9023b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a6186b9f2d40c1a6a1e0f8b38f64a3, user_id=2890ef7cdb504a9bb6bc47d419a234f8, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:19.671618 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243310a-621f-42b5-ab69-9d1f4f9023b6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:19.727760 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 913/1819] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:19 2022] PUT /identity/v3/projects/d8a6186b9f2d40c1a6a1e0f8b38f64a3/users/2890ef7cdb504a9bb6bc47d419a234f8/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:19.736670 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b748f-a5a4-43aa-a449-da48747a2380 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:19.736670 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b748f-a5a4-43aa-a449-da48747a2380 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:19.736670 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b748f-a5a4-43aa-a449-da48747a2380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:19.791107 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e40b748f-a5a4-43aa-a449-da48747a2380 None None] MFA Rules not processed for user `2890ef7cdb504a9bb6bc47d419a234f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:19.792454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e40b748f-a5a4-43aa-a449-da48747a2380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:19.916189 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 907/1820] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:19.930093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243310a-621f-42b5-ab69-9d1f4f9023b6 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:19.931878 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2243310a-621f-42b5-ab69-9d1f4f9023b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:20.047359 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243310a-621f-42b5-ab69-9d1f4f9023b6 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:20.057042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51878679-3ea9-4a71-9e44-bdfbe59b1f1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:20.058336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51878679-3ea9-4a71-9e44-bdfbe59b1f1a service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:20.059038 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51878679-3ea9-4a71-9e44-bdfbe59b1f1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:20.059678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51878679-3ea9-4a71-9e44-bdfbe59b1f1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:20.072810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51878679-3ea9-4a71-9e44-bdfbe59b1f1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:20.075260 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51878679-3ea9-4a71-9e44-bdfbe59b1f1a service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:20.089555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 914/1821] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:20.262302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b748f-a5a4-43aa-a449-da48747a2380 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:20.263947 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e40b748f-a5a4-43aa-a449-da48747a2380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:20.372146 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b748f-a5a4-43aa-a449-da48747a2380 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:20.382243 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c6822-9758-47eb-97d0-896115dc0906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:20.383108 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c6822-9758-47eb-97d0-896115dc0906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:20.383590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c6822-9758-47eb-97d0-896115dc0906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:20.383590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c6822-9758-47eb-97d0-896115dc0906 admin admin] PATH_INFO: `/v3/projects/2978768e24234d7fabc46f44299bcaa5` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:20.384870 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c6822-9758-47eb-97d0-896115dc0906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2978768e24234d7fabc46f44299bcaa5)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:20.387064 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c6822-9758-47eb-97d0-896115dc0906 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:20.450904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:20.459268 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8f9c6822-9758-47eb-97d0-896115dc0906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2978768e24234d7fabc46f44299bcaa5', 'request_id': 'req-8f9c6822-9758-47eb-97d0-896115dc0906'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:20.469392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8f9c6822-9758-47eb-97d0-896115dc0906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2978768e24234d7fabc46f44299bcaa5', 'request_id': 'req-8f9c6822-9758-47eb-97d0-896115dc0906'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:20.483095 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8f9c6822-9758-47eb-97d0-896115dc0906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2978768e24234d7fabc46f44299bcaa5', 'request_id': 'req-8f9c6822-9758-47eb-97d0-896115dc0906'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:20.520072 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8f9c6822-9758-47eb-97d0-896115dc0906 admin admin] The token cache is being invalidate because project 2978768e24234d7fabc46f44299bcaa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:20.521441 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8f9c6822-9758-47eb-97d0-896115dc0906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:20.522663 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8f9c6822-9758-47eb-97d0-896115dc0906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:20.527319 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 908/1822] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:20 2022] DELETE /identity/v3/projects/2978768e24234d7fabc46f44299bcaa5 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:20.531812 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51878679-3ea9-4a71-9e44-bdfbe59b1f1a service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:20.545406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51878679-3ea9-4a71-9e44-bdfbe59b1f1a service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:20.568776 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:20.662528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413fbae-d29f-4a2b-80c3-f3e8c23b69bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:20.663188 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413fbae-d29f-4a2b-80c3-f3e8c23b69bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:20.663470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413fbae-d29f-4a2b-80c3-f3e8c23b69bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:20.663744 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413fbae-d29f-4a2b-80c3-f3e8c23b69bf admin admin] PATH_INFO: `/v3/users/7ff9400855574ca0878b7569536454c2` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:20.664974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413fbae-d29f-4a2b-80c3-f3e8c23b69bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ff9400855574ca0878b7569536454c2)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:20.666954 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413fbae-d29f-4a2b-80c3-f3e8c23b69bf admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:20.780975 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c6822-9758-47eb-97d0-896115dc0906 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:20.784957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8f9c6822-9758-47eb-97d0-896115dc0906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:20.812896 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5413fbae-d29f-4a2b-80c3-f3e8c23b69bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ff9400855574ca0878b7569536454c2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:20.813590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5413fbae-d29f-4a2b-80c3-f3e8c23b69bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ff9400855574ca0878b7569536454c2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:20.825992 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5413fbae-d29f-4a2b-80c3-f3e8c23b69bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ff9400855574ca0878b7569536454c2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:20.836320 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5413fbae-d29f-4a2b-80c3-f3e8c23b69bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ff9400855574ca0878b7569536454c2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:20.850950 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5413fbae-d29f-4a2b-80c3-f3e8c23b69bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ff9400855574ca0878b7569536454c2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:20.857578 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 915/1823] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:20 2022] DELETE /identity/v3/users/7ff9400855574ca0878b7569536454c2 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:20.890893 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c6822-9758-47eb-97d0-896115dc0906 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:20.964831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf4f96-29b6-4110-ae2e-c988b52c29ff tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2890ef7cdb504a9bb6bc47d419a234f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8a6186b9f2d40c1a6a1e0f8b38f64a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:20.965873 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf4f96-29b6-4110-ae2e-c988b52c29ff tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:20.966490 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf4f96-29b6-4110-ae2e-c988b52c29ff tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:20.966943 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf4f96-29b6-4110-ae2e-c988b52c29ff tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:20.968130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf4f96-29b6-4110-ae2e-c988b52c29ff tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:20.968661 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf4f96-29b6-4110-ae2e-c988b52c29ff tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:20.972080 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf4f96-29b6-4110-ae2e-c988b52c29ff tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:20.996298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 909/1824] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:56:21.006491 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413fbae-d29f-4a2b-80c3-f3e8c23b69bf admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:21.011708 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5413fbae-d29f-4a2b-80c3-f3e8c23b69bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:21.030689 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413fbae-d29f-4a2b-80c3-f3e8c23b69bf admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:21.035728 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439db38-e4da-41b3-a774-57b5a96ebb55 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2890ef7cdb504a9bb6bc47d419a234f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8a6186b9f2d40c1a6a1e0f8b38f64a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:21.036416 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439db38-e4da-41b3-a774-57b5a96ebb55 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:21.036704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439db38-e4da-41b3-a774-57b5a96ebb55 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:21.037124 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439db38-e4da-41b3-a774-57b5a96ebb55 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:21.038214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439db38-e4da-41b3-a774-57b5a96ebb55 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:21.044542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439db38-e4da-41b3-a774-57b5a96ebb55 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:21.081341 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 916/1825] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:21 2022] POST /identity/v3/projects => generated 354 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:21.087226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf4f96-29b6-4110-ae2e-c988b52c29ff tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:21.103036 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf4f96-29b6-4110-ae2e-c988b52c29ff tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:21.195131 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e3d24-f71d-4fa0-bfae-2e8e1a96b4f3 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2890ef7cdb504a9bb6bc47d419a234f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8a6186b9f2d40c1a6a1e0f8b38f64a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:21.196020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e3d24-f71d-4fa0-bfae-2e8e1a96b4f3 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:21.196500 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e3d24-f71d-4fa0-bfae-2e8e1a96b4f3 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:21.196972 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e3d24-f71d-4fa0-bfae-2e8e1a96b4f3 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:21.198147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e3d24-f71d-4fa0-bfae-2e8e1a96b4f3 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:21.198485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e3d24-f71d-4fa0-bfae-2e8e1a96b4f3 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:21.200693 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e3d24-f71d-4fa0-bfae-2e8e1a96b4f3 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:21.211960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:21.215685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439db38-e4da-41b3-a774-57b5a96ebb55 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:21.218201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0439db38-e4da-41b3-a774-57b5a96ebb55 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:21.222960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 910/1826] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:56:21.343385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e3d24-f71d-4fa0-bfae-2e8e1a96b4f3 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:21.358487 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439db38-e4da-41b3-a774-57b5a96ebb55 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:21.362232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e3d24-f71d-4fa0-bfae-2e8e1a96b4f3 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:21.366707 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1848fe-b141-4c22-a5af-836cd2ddd846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:21.368080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1848fe-b141-4c22-a5af-836cd2ddd846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:21.368394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1848fe-b141-4c22-a5af-836cd2ddd846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:21.368668 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1848fe-b141-4c22-a5af-836cd2ddd846 admin admin] PATH_INFO: `/v3/projects/8654e1a62bb24fa8904c22704dd5c7c2` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:21.369904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1848fe-b141-4c22-a5af-836cd2ddd846 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8654e1a62bb24fa8904c22704dd5c7c2)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:21.370956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-af1848fe-b141-4c22-a5af-836cd2ddd846 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 08:56:21.370956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Aug 24 08:56:21.370956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: Aug 24 08:56:21.372406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1848fe-b141-4c22-a5af-836cd2ddd846 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:21.372961 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4405457-8222-41b9-a936-6ec859ff1ee5 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2890ef7cdb504a9bb6bc47d419a234f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8a6186b9f2d40c1a6a1e0f8b38f64a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:21.374693 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4405457-8222-41b9-a936-6ec859ff1ee5 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:21.375319 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4405457-8222-41b9-a936-6ec859ff1ee5 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:21.375762 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4405457-8222-41b9-a936-6ec859ff1ee5 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] PATH_INFO: `/v3/projects/190f5bcf05704a22999fa7255f28e2ad` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:21.378046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4405457-8222-41b9-a936-6ec859ff1ee5 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] RBAC: Authorizing `identity:delete_project(project_id=190f5bcf05704a22999fa7255f28e2ad)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:21.380283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4405457-8222-41b9-a936-6ec859ff1ee5 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:21.424758 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:21.467772 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a4405457-8222-41b9-a936-6ec859ff1ee5 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '190f5bcf05704a22999fa7255f28e2ad', 'request_id': 'req-a4405457-8222-41b9-a936-6ec859ff1ee5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:21.477903 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-af1848fe-b141-4c22-a5af-836cd2ddd846 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8654e1a62bb24fa8904c22704dd5c7c2', 'request_id': 'req-af1848fe-b141-4c22-a5af-836cd2ddd846'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:21.480170 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a4405457-8222-41b9-a936-6ec859ff1ee5 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '190f5bcf05704a22999fa7255f28e2ad', 'request_id': 'req-a4405457-8222-41b9-a936-6ec859ff1ee5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:21.486458 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-af1848fe-b141-4c22-a5af-836cd2ddd846 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8654e1a62bb24fa8904c22704dd5c7c2', 'request_id': 'req-af1848fe-b141-4c22-a5af-836cd2ddd846'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:21.489878 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a4405457-8222-41b9-a936-6ec859ff1ee5 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '190f5bcf05704a22999fa7255f28e2ad', 'request_id': 'req-a4405457-8222-41b9-a936-6ec859ff1ee5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:21.497463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-af1848fe-b141-4c22-a5af-836cd2ddd846 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8654e1a62bb24fa8904c22704dd5c7c2', 'request_id': 'req-af1848fe-b141-4c22-a5af-836cd2ddd846'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:21.508757 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a4405457-8222-41b9-a936-6ec859ff1ee5 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] The token cache is being invalidate because project 190f5bcf05704a22999fa7255f28e2ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:21.509116 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a4405457-8222-41b9-a936-6ec859ff1ee5 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:21.509888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a4405457-8222-41b9-a936-6ec859ff1ee5 tempest-VolumeQuotaClassesTest-41867566 tempest-VolumeQuotaClassesTest-41867566-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:21.511535 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 911/1827] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:21 2022] DELETE /identity/v3/projects/190f5bcf05704a22999fa7255f28e2ad => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:21.541927 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-af1848fe-b141-4c22-a5af-836cd2ddd846 admin admin] The token cache is being invalidate because project 8654e1a62bb24fa8904c22704dd5c7c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:21.543464 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-af1848fe-b141-4c22-a5af-836cd2ddd846 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:21.544504 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-af1848fe-b141-4c22-a5af-836cd2ddd846 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:21.547908 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 917/1828] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:21 2022] DELETE /identity/v3/projects/8654e1a62bb24fa8904c22704dd5c7c2 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:21.554792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fd626-935f-47c8-a6f8-2dba9946e343 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:21.555434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fd626-935f-47c8-a6f8-2dba9946e343 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:21.555947 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fd626-935f-47c8-a6f8-2dba9946e343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:21.583749 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-a59fd626-935f-47c8-a6f8-2dba9946e343 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:21.585306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a59fd626-935f-47c8-a6f8-2dba9946e343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:21.685714 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 912/1829] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:21.694706 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d4243-b794-4cc2-a9b2-c4c60df8d424 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:21.695191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d4243-b794-4cc2-a9b2-c4c60df8d424 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:21.695637 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d4243-b794-4cc2-a9b2-c4c60df8d424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:21.729130 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5a6d4243-b794-4cc2-a9b2-c4c60df8d424 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:21.730565 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5a6d4243-b794-4cc2-a9b2-c4c60df8d424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:21.742896 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 918/1830] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:21.747401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fd626-935f-47c8-a6f8-2dba9946e343 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:21.761766 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fd626-935f-47c8-a6f8-2dba9946e343 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:21.768265 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505b506-87e3-40ce-8f77-15a11c8b0fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:21.768950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505b506-87e3-40ce-8f77-15a11c8b0fbb admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:21.769234 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505b506-87e3-40ce-8f77-15a11c8b0fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:21.769512 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505b506-87e3-40ce-8f77-15a11c8b0fbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:21.772929 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505b506-87e3-40ce-8f77-15a11c8b0fbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:21.772929 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505b506-87e3-40ce-8f77-15a11c8b0fbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:21.772929 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505b506-87e3-40ce-8f77-15a11c8b0fbb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:21.789596 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 913/1831] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:56:21.794884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d4243-b794-4cc2-a9b2-c4c60df8d424 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:21.812777 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d4243-b794-4cc2-a9b2-c4c60df8d424 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:21.818917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c8407-a662-4e8e-b66c-b97a77e70739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:21.819553 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c8407-a662-4e8e-b66c-b97a77e70739 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:21.819898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c8407-a662-4e8e-b66c-b97a77e70739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:21.820210 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c8407-a662-4e8e-b66c-b97a77e70739 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:21.821019 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c8407-a662-4e8e-b66c-b97a77e70739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:21.823241 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c8407-a662-4e8e-b66c-b97a77e70739 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:21.850263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505b506-87e3-40ce-8f77-15a11c8b0fbb admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:21.853218 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e505b506-87e3-40ce-8f77-15a11c8b0fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:21.873512 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 919/1832] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:21 2022] POST /identity/v3/projects => generated 375 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:21.880907 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c8407-a662-4e8e-b66c-b97a77e70739 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:21.885251 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505b506-87e3-40ce-8f77-15a11c8b0fbb admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:22.043065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c8407-a662-4e8e-b66c-b97a77e70739 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:22.048181 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d950a-aac3-4208-b480-ee00f396f498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:22.049869 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d950a-aac3-4208-b480-ee00f396f498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:22.050267 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d950a-aac3-4208-b480-ee00f396f498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:22.050649 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d950a-aac3-4208-b480-ee00f396f498 admin admin] PATH_INFO: `/v3/users/b50eadbcc7fd4257b6c895edccd6fd1d` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:22.051581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1a808-1bea-4eb0-9e18-407affa2faa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:22.052370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1a808-1bea-4eb0-9e18-407affa2faa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:22.052786 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1a808-1bea-4eb0-9e18-407affa2faa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:22.053141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1a808-1bea-4eb0-9e18-407affa2faa6 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:22.054073 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1a808-1bea-4eb0-9e18-407affa2faa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:22.055095 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d950a-aac3-4208-b480-ee00f396f498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b50eadbcc7fd4257b6c895edccd6fd1d)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:22.056312 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1a808-1bea-4eb0-9e18-407affa2faa6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:22.059467 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d950a-aac3-4208-b480-ee00f396f498 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:22.086962 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 920/1833] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:21 2022] POST /identity/v3/users => generated 288 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:22.088663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1a808-1bea-4eb0-9e18-407affa2faa6 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:22.092162 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:22.102131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1a808-1bea-4eb0-9e18-407affa2faa6 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:22.109392 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371426c3-ee76-4e1b-8c20-98f5b3be76fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:22.110026 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371426c3-ee76-4e1b-8c20-98f5b3be76fd admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:22.110308 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371426c3-ee76-4e1b-8c20-98f5b3be76fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:22.111515 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371426c3-ee76-4e1b-8c20-98f5b3be76fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:22.111515 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371426c3-ee76-4e1b-8c20-98f5b3be76fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:22.113976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371426c3-ee76-4e1b-8c20-98f5b3be76fd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:22.128892 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 921/1834] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:56:22.133328 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371426c3-ee76-4e1b-8c20-98f5b3be76fd admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:22.174260 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371426c3-ee76-4e1b-8c20-98f5b3be76fd admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:22.181071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01d674-d92b-45a8-876c-f6e306d628a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:22.181702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01d674-d92b-45a8-876c-f6e306d628a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:22.181978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01d674-d92b-45a8-876c-f6e306d628a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:22.182248 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01d674-d92b-45a8-876c-f6e306d628a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:22.182992 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01d674-d92b-45a8-876c-f6e306d628a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:22.184653 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01d674-d92b-45a8-876c-f6e306d628a2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:22.184997 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6b1d950a-aac3-4208-b480-ee00f396f498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b50eadbcc7fd4257b6c895edccd6fd1d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:22.199235 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 922/1835] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:56:22.200991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6b1d950a-aac3-4208-b480-ee00f396f498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b50eadbcc7fd4257b6c895edccd6fd1d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:22.204133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01d674-d92b-45a8-876c-f6e306d628a2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:22.215637 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01d674-d92b-45a8-876c-f6e306d628a2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:22.221998 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d1c0e-828c-451a-9e25-b333eb2b62a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:22.222693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d1c0e-828c-451a-9e25-b333eb2b62a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:22.222969 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d1c0e-828c-451a-9e25-b333eb2b62a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:22.223245 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d1c0e-828c-451a-9e25-b333eb2b62a4 admin admin] PATH_INFO: `/v3/projects/532d1831dfbb43dfba6f3f20ebc8153a/users/91ec6d1fef304f608af68eaf67f1dbff/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:22.227769 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6b1d950a-aac3-4208-b480-ee00f396f498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b50eadbcc7fd4257b6c895edccd6fd1d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:22.243115 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6b1d950a-aac3-4208-b480-ee00f396f498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b50eadbcc7fd4257b6c895edccd6fd1d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:22.244462 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6b1d950a-aac3-4208-b480-ee00f396f498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b50eadbcc7fd4257b6c895edccd6fd1d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:22.245851 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d1c0e-828c-451a-9e25-b333eb2b62a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=532d1831dfbb43dfba6f3f20ebc8153a, user_id=91ec6d1fef304f608af68eaf67f1dbff, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:22.248727 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d1c0e-828c-451a-9e25-b333eb2b62a4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:22.251552 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 914/1836] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:21 2022] DELETE /identity/v3/users/b50eadbcc7fd4257b6c895edccd6fd1d => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:22.256601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d8583-b808-4fc9-bb93-d60fd0becef4 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:22.256919 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d8583-b808-4fc9-bb93-d60fd0becef4 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:22.257268 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d8583-b808-4fc9-bb93-d60fd0becef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:22.270680 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 923/1837] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:22 2022] PUT /identity/v3/projects/532d1831dfbb43dfba6f3f20ebc8153a/users/91ec6d1fef304f608af68eaf67f1dbff/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:22.289626 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3676cf-683e-4e18-b151-9287b1454448 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:22.290122 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3676cf-683e-4e18-b151-9287b1454448 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:22.290575 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3676cf-683e-4e18-b151-9287b1454448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:22.297110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-6f1d8583-b808-4fc9-bb93-d60fd0becef4 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:22.299045 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6f1d8583-b808-4fc9-bb93-d60fd0becef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:22.334934 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7b3676cf-683e-4e18-b151-9287b1454448 None None] MFA Rules not processed for user `91ec6d1fef304f608af68eaf67f1dbff`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:22.336332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7b3676cf-683e-4e18-b151-9287b1454448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:22.347270 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:22.402133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 915/1838] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:22.422884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d8583-b808-4fc9-bb93-d60fd0becef4 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:22.425528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6f1d8583-b808-4fc9-bb93-d60fd0becef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:22.456072 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 924/1839] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:22.460782 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3676cf-683e-4e18-b151-9287b1454448 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:22.462230 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7b3676cf-683e-4e18-b151-9287b1454448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:22.481719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3676cf-683e-4e18-b151-9287b1454448 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:22.487149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181ec06-9d6c-4019-80df-3ee3c47309bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:22.487777 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181ec06-9d6c-4019-80df-3ee3c47309bc admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:22.488349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181ec06-9d6c-4019-80df-3ee3c47309bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:22.488724 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181ec06-9d6c-4019-80df-3ee3c47309bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:22.489279 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181ec06-9d6c-4019-80df-3ee3c47309bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:22.491207 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181ec06-9d6c-4019-80df-3ee3c47309bc admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:22.504126 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d8583-b808-4fc9-bb93-d60fd0becef4 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:22.509494 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7f4d6-5169-41ae-8405-d337a149207c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:22.510579 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7f4d6-5169-41ae-8405-d337a149207c service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:22.511055 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7f4d6-5169-41ae-8405-d337a149207c service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:22.511566 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7f4d6-5169-41ae-8405-d337a149207c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:22.526646 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d7f4d6-5169-41ae-8405-d337a149207c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:22.528247 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d7f4d6-5169-41ae-8405-d337a149207c service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:22.531287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 925/1840] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:22 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:22.536747 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181ec06-9d6c-4019-80df-3ee3c47309bc admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:22.542522 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 916/1841] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:56:22.627167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181ec06-9d6c-4019-80df-3ee3c47309bc admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:22.634206 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8b733-5378-4f1b-aeba-6e86f3d1e460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:22.637563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8b733-5378-4f1b-aeba-6e86f3d1e460 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:22.637563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8b733-5378-4f1b-aeba-6e86f3d1e460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:22.637563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8b733-5378-4f1b-aeba-6e86f3d1e460 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:22.637563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8b733-5378-4f1b-aeba-6e86f3d1e460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:22.638561 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8b733-5378-4f1b-aeba-6e86f3d1e460 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:22.666941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 926/1842] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:22 2022] POST /identity/v3/users => generated 295 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:22.673797 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7f4d6-5169-41ae-8405-d337a149207c service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:22.707949 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7f4d6-5169-41ae-8405-d337a149207c service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:22.718082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe20e83-e1c1-4957-9ed4-a187d4a4a81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:22.718845 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe20e83-e1c1-4957-9ed4-a187d4a4a81d admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:22.719128 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe20e83-e1c1-4957-9ed4-a187d4a4a81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:22.719398 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe20e83-e1c1-4957-9ed4-a187d4a4a81d admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:22.720133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe20e83-e1c1-4957-9ed4-a187d4a4a81d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:22.724034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe20e83-e1c1-4957-9ed4-a187d4a4a81d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:22.784231 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 917/1843] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:56:22.785267 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8b733-5378-4f1b-aeba-6e86f3d1e460 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:22.805518 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8b733-5378-4f1b-aeba-6e86f3d1e460 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:22.815864 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d25b12-588b-4780-8eba-47bc1aa91842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:22.816632 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d25b12-588b-4780-8eba-47bc1aa91842 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:22.816937 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d25b12-588b-4780-8eba-47bc1aa91842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:22.817645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d25b12-588b-4780-8eba-47bc1aa91842 admin admin] PATH_INFO: `/v3/projects/8f239f1a387e48ee9ddaa381e3f4882b/users/5461439bc88840619ef97302ed48c7a6/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:22.856026 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d25b12-588b-4780-8eba-47bc1aa91842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f239f1a387e48ee9ddaa381e3f4882b, user_id=5461439bc88840619ef97302ed48c7a6, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:22.859725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d25b12-588b-4780-8eba-47bc1aa91842 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:22.883830 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 927/1844] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:22 2022] PUT /identity/v3/projects/8f239f1a387e48ee9ddaa381e3f4882b/users/5461439bc88840619ef97302ed48c7a6/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:22.887938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe20e83-e1c1-4957-9ed4-a187d4a4a81d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:22.910839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe20e83-e1c1-4957-9ed4-a187d4a4a81d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:22.956902 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:23.033832 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334aefe-718b-4d90-826f-fac6efa80ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:23.035093 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334aefe-718b-4d90-826f-fac6efa80ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:23.035567 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334aefe-718b-4d90-826f-fac6efa80ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:23.036275 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334aefe-718b-4d90-826f-fac6efa80ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:23.038439 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334aefe-718b-4d90-826f-fac6efa80ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:23.044056 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334aefe-718b-4d90-826f-fac6efa80ae9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:23.064872 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 918/1845] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:56:23.066264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d25b12-588b-4780-8eba-47bc1aa91842 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:23.094971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d25b12-588b-4780-8eba-47bc1aa91842 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:23.106994 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10452a8a-8587-402e-87cd-0d5de57c9b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:23.108115 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10452a8a-8587-402e-87cd-0d5de57c9b6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:23.108646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10452a8a-8587-402e-87cd-0d5de57c9b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:23.109095 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10452a8a-8587-402e-87cd-0d5de57c9b6f admin admin] PATH_INFO: `/v3/projects/8f239f1a387e48ee9ddaa381e3f4882b/users/5461439bc88840619ef97302ed48c7a6/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:23.113265 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10452a8a-8587-402e-87cd-0d5de57c9b6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f239f1a387e48ee9ddaa381e3f4882b, user_id=5461439bc88840619ef97302ed48c7a6, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:23.118447 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10452a8a-8587-402e-87cd-0d5de57c9b6f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:23.136943 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 928/1846] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:23 2022] PUT /identity/v3/projects/8f239f1a387e48ee9ddaa381e3f4882b/users/5461439bc88840619ef97302ed48c7a6/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:23.151662 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d87521-21b5-4f1f-b70c-98c172a8ec44 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:23.152227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d87521-21b5-4f1f-b70c-98c172a8ec44 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:23.152741 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d87521-21b5-4f1f-b70c-98c172a8ec44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:23.201182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-a8d87521-21b5-4f1f-b70c-98c172a8ec44 None None] MFA Rules not processed for user `5461439bc88840619ef97302ed48c7a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:23.202644 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a8d87521-21b5-4f1f-b70c-98c172a8ec44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:23.325155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 919/1847] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:23 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:23.340024 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10452a8a-8587-402e-87cd-0d5de57c9b6f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:23.342910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-10452a8a-8587-402e-87cd-0d5de57c9b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:23.420130 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:23.450469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10452a8a-8587-402e-87cd-0d5de57c9b6f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:23.458371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a421667-535c-49a0-b57a-701eede2aafd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:23.459760 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a421667-535c-49a0-b57a-701eede2aafd service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:23.460071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a421667-535c-49a0-b57a-701eede2aafd service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:23.460629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a421667-535c-49a0-b57a-701eede2aafd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:23.557610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a421667-535c-49a0-b57a-701eede2aafd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:23.559403 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a421667-535c-49a0-b57a-701eede2aafd service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:23.576297 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 929/1848] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:24.160592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d87521-21b5-4f1f-b70c-98c172a8ec44 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:24.162825 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a8d87521-21b5-4f1f-b70c-98c172a8ec44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:24.310009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d87521-21b5-4f1f-b70c-98c172a8ec44 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:24.321955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f66dd9-5545-4557-8d99-b7238825df0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:24.322945 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f66dd9-5545-4557-8d99-b7238825df0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:24.323239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f66dd9-5545-4557-8d99-b7238825df0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:24.323528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f66dd9-5545-4557-8d99-b7238825df0f admin admin] PATH_INFO: `/v3/projects/a711cf5e0dae48aba0509bd011be4261` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:24.335520 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f66dd9-5545-4557-8d99-b7238825df0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a711cf5e0dae48aba0509bd011be4261)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:24.340333 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f66dd9-5545-4557-8d99-b7238825df0f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:24.421079 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:24.426385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-30f66dd9-5545-4557-8d99-b7238825df0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a711cf5e0dae48aba0509bd011be4261', 'request_id': 'req-30f66dd9-5545-4557-8d99-b7238825df0f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:24.446136 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-30f66dd9-5545-4557-8d99-b7238825df0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a711cf5e0dae48aba0509bd011be4261', 'request_id': 'req-30f66dd9-5545-4557-8d99-b7238825df0f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:24.464358 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-30f66dd9-5545-4557-8d99-b7238825df0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a711cf5e0dae48aba0509bd011be4261', 'request_id': 'req-30f66dd9-5545-4557-8d99-b7238825df0f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:24.486403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-30f66dd9-5545-4557-8d99-b7238825df0f admin admin] The token cache is being invalidate because project a711cf5e0dae48aba0509bd011be4261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:24.486749 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-30f66dd9-5545-4557-8d99-b7238825df0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:24.487463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-30f66dd9-5545-4557-8d99-b7238825df0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:24.494719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 920/1849] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:24 2022] DELETE /identity/v3/projects/a711cf5e0dae48aba0509bd011be4261 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:24.496377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a421667-535c-49a0-b57a-701eede2aafd service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:24.518839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a421667-535c-49a0-b57a-701eede2aafd service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:24.611207 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf015caa-f8fc-4cb8-b8f6-2fe92f82f82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:24.611887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf015caa-f8fc-4cb8-b8f6-2fe92f82f82e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:24.612199 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf015caa-f8fc-4cb8-b8f6-2fe92f82f82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:24.612475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf015caa-f8fc-4cb8-b8f6-2fe92f82f82e admin admin] PATH_INFO: `/v3/users/2890ef7cdb504a9bb6bc47d419a234f8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:24.613955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf015caa-f8fc-4cb8-b8f6-2fe92f82f82e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2890ef7cdb504a9bb6bc47d419a234f8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:24.615931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf015caa-f8fc-4cb8-b8f6-2fe92f82f82e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:24.681673 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cf015caa-f8fc-4cb8-b8f6-2fe92f82f82e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2890ef7cdb504a9bb6bc47d419a234f8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:24.682515 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cf015caa-f8fc-4cb8-b8f6-2fe92f82f82e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2890ef7cdb504a9bb6bc47d419a234f8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:24.692062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cf015caa-f8fc-4cb8-b8f6-2fe92f82f82e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2890ef7cdb504a9bb6bc47d419a234f8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:24.701509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cf015caa-f8fc-4cb8-b8f6-2fe92f82f82e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2890ef7cdb504a9bb6bc47d419a234f8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:24.717146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cf015caa-f8fc-4cb8-b8f6-2fe92f82f82e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2890ef7cdb504a9bb6bc47d419a234f8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:24.728398 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 930/1850] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:24 2022] DELETE /identity/v3/users/2890ef7cdb504a9bb6bc47d419a234f8 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:25.253628 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f66dd9-5545-4557-8d99-b7238825df0f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:25.255183 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-30f66dd9-5545-4557-8d99-b7238825df0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:25.293940 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f66dd9-5545-4557-8d99-b7238825df0f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:25.391566 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a038b8c-58c0-4139-b446-2212b84ae3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:25.392806 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a038b8c-58c0-4139-b446-2212b84ae3f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:25.393087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a038b8c-58c0-4139-b446-2212b84ae3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:25.393364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a038b8c-58c0-4139-b446-2212b84ae3f3 admin admin] PATH_INFO: `/v3/projects/d8a6186b9f2d40c1a6a1e0f8b38f64a3` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:25.394569 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a038b8c-58c0-4139-b446-2212b84ae3f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8a6186b9f2d40c1a6a1e0f8b38f64a3)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:25.396582 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a038b8c-58c0-4139-b446-2212b84ae3f3 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:25.478962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1a038b8c-58c0-4139-b446-2212b84ae3f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8a6186b9f2d40c1a6a1e0f8b38f64a3', 'request_id': 'req-1a038b8c-58c0-4139-b446-2212b84ae3f3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:25.496420 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1a038b8c-58c0-4139-b446-2212b84ae3f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8a6186b9f2d40c1a6a1e0f8b38f64a3', 'request_id': 'req-1a038b8c-58c0-4139-b446-2212b84ae3f3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:25.506402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1a038b8c-58c0-4139-b446-2212b84ae3f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8a6186b9f2d40c1a6a1e0f8b38f64a3', 'request_id': 'req-1a038b8c-58c0-4139-b446-2212b84ae3f3'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:25.523637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1a038b8c-58c0-4139-b446-2212b84ae3f3 admin admin] The token cache is being invalidate because project d8a6186b9f2d40c1a6a1e0f8b38f64a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:25.524048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1a038b8c-58c0-4139-b446-2212b84ae3f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:25.524861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1a038b8c-58c0-4139-b446-2212b84ae3f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:25.526672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 921/1851] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:25 2022] DELETE /identity/v3/projects/d8a6186b9f2d40c1a6a1e0f8b38f64a3 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:25.532178 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2a6d1-5671-4854-a631-01027200677e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:25.532820 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2a6d1-5671-4854-a631-01027200677e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:25.533622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2a6d1-5671-4854-a631-01027200677e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:25.573178 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-fff2a6d1-5671-4854-a631-01027200677e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:25.575240 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fff2a6d1-5671-4854-a631-01027200677e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:25.656461 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:25.728007 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 931/1852] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:25.743841 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c903f5-a7de-4a18-8b19-e9d146793c02 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:25.743841 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c903f5-a7de-4a18-8b19-e9d146793c02 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:25.743841 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c903f5-a7de-4a18-8b19-e9d146793c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:25.766917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-36c903f5-a7de-4a18-8b19-e9d146793c02 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:25.773388 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-36c903f5-a7de-4a18-8b19-e9d146793c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:25.805546 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 922/1853] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:25.809837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2a6d1-5671-4854-a631-01027200677e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:25.822037 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2a6d1-5671-4854-a631-01027200677e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:25.826827 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a251745-b9a7-4b79-a2c3-99d1c88f686a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:25.827448 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a251745-b9a7-4b79-a2c3-99d1c88f686a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:25.827726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a251745-b9a7-4b79-a2c3-99d1c88f686a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:25.827998 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a251745-b9a7-4b79-a2c3-99d1c88f686a admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:25.828687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a251745-b9a7-4b79-a2c3-99d1c88f686a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:25.828993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a251745-b9a7-4b79-a2c3-99d1c88f686a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:25.831260 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a251745-b9a7-4b79-a2c3-99d1c88f686a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:25.843368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 932/1854] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:56:25.848309 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c903f5-a7de-4a18-8b19-e9d146793c02 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:25.857777 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c903f5-a7de-4a18-8b19-e9d146793c02 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:25.864201 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577b7c2-1ddb-41bc-8e76-284b0be55177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:25.864840 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577b7c2-1ddb-41bc-8e76-284b0be55177 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:25.865221 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577b7c2-1ddb-41bc-8e76-284b0be55177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:25.865468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577b7c2-1ddb-41bc-8e76-284b0be55177 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:25.866154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577b7c2-1ddb-41bc-8e76-284b0be55177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:25.868147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577b7c2-1ddb-41bc-8e76-284b0be55177 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:25.905436 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 923/1855] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:25 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:25.910802 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a251745-b9a7-4b79-a2c3-99d1c88f686a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:25.924184 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a251745-b9a7-4b79-a2c3-99d1c88f686a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:25.991459 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519544e-634c-4e38-b24e-f2d8610e7acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:25.992083 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519544e-634c-4e38-b24e-f2d8610e7acd admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:25.992386 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519544e-634c-4e38-b24e-f2d8610e7acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:25.992659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519544e-634c-4e38-b24e-f2d8610e7acd admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:25.993290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519544e-634c-4e38-b24e-f2d8610e7acd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:25.995218 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519544e-634c-4e38-b24e-f2d8610e7acd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:26.019583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 933/1856] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:25 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:26.024013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577b7c2-1ddb-41bc-8e76-284b0be55177 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:26.035060 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577b7c2-1ddb-41bc-8e76-284b0be55177 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:26.039861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05a8cc-8798-4507-8142-a0c3b5f55f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:26.040511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05a8cc-8798-4507-8142-a0c3b5f55f84 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.040789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05a8cc-8798-4507-8142-a0c3b5f55f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.041061 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05a8cc-8798-4507-8142-a0c3b5f55f84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.041734 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05a8cc-8798-4507-8142-a0c3b5f55f84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:26.043319 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05a8cc-8798-4507-8142-a0c3b5f55f84 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:26.057067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 924/1857] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:26.062170 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519544e-634c-4e38-b24e-f2d8610e7acd admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:26.073840 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519544e-634c-4e38-b24e-f2d8610e7acd admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:26.078960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80419d45-b5d8-4807-ae4d-62a3c33fe593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:26.080418 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80419d45-b5d8-4807-ae4d-62a3c33fe593 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.080732 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80419d45-b5d8-4807-ae4d-62a3c33fe593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.081061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80419d45-b5d8-4807-ae4d-62a3c33fe593 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.081676 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80419d45-b5d8-4807-ae4d-62a3c33fe593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:26.083354 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80419d45-b5d8-4807-ae4d-62a3c33fe593 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:26.096000 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 934/1858] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:26.100655 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05a8cc-8798-4507-8142-a0c3b5f55f84 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:26.112295 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05a8cc-8798-4507-8142-a0c3b5f55f84 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:26.118100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7ecc1-9d2c-45c8-9bc2-b3f3b5f05e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:26.118834 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7ecc1-9d2c-45c8-9bc2-b3f3b5f05e56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.119112 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7ecc1-9d2c-45c8-9bc2-b3f3b5f05e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.119381 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7ecc1-9d2c-45c8-9bc2-b3f3b5f05e56 admin admin] PATH_INFO: `/v3/projects/3dfa220132114b60b3607e8b4687bda5/users/df03267bb8c646d1b2ffc1706e45eef8/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.145376 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a7ecc1-9d2c-45c8-9bc2-b3f3b5f05e56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dfa220132114b60b3607e8b4687bda5, user_id=df03267bb8c646d1b2ffc1706e45eef8, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:26.148190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a7ecc1-9d2c-45c8-9bc2-b3f3b5f05e56 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:26.159153 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:26.165606 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 925/1859] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:26 2022] PUT /identity/v3/projects/3dfa220132114b60b3607e8b4687bda5/users/df03267bb8c646d1b2ffc1706e45eef8/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:26.180467 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ec681-d585-42d2-837a-23b84dd79f5b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.180844 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ec681-d585-42d2-837a-23b84dd79f5b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.181260 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ec681-d585-42d2-837a-23b84dd79f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.214580 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f35ec681-d585-42d2-837a-23b84dd79f5b None None] MFA Rules not processed for user `df03267bb8c646d1b2ffc1706e45eef8`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:26.215877 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f35ec681-d585-42d2-837a-23b84dd79f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:26.288720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 935/1860] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:26.293242 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7ecc1-9d2c-45c8-9bc2-b3f3b5f05e56 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:26.387198 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7ecc1-9d2c-45c8-9bc2-b3f3b5f05e56 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:26.393935 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a605db9-d1d6-4c29-a69c-a53a20e5b0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:26.394581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a605db9-d1d6-4c29-a69c-a53a20e5b0f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.394861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a605db9-d1d6-4c29-a69c-a53a20e5b0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.395131 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a605db9-d1d6-4c29-a69c-a53a20e5b0f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.395792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a605db9-d1d6-4c29-a69c-a53a20e5b0f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:26.397859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a605db9-d1d6-4c29-a69c-a53a20e5b0f6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:26.432183 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 926/1861] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:26 2022] POST /identity/v3/projects => generated 387 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:26.436888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ec681-d585-42d2-837a-23b84dd79f5b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:26.465084 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ec681-d585-42d2-837a-23b84dd79f5b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:26.479009 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:26.537200 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd7247-2666-4653-b767-39acd6ba7bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:26.537905 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd7247-2666-4653-b767-39acd6ba7bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.537995 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd7247-2666-4653-b767-39acd6ba7bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.538382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd7247-2666-4653-b767-39acd6ba7bd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.540395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd7247-2666-4653-b767-39acd6ba7bd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:26.542567 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd7247-2666-4653-b767-39acd6ba7bd0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:26.572265 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 936/1862] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:26 2022] POST /identity/v3/users => generated 294 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:56:26.576215 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a605db9-d1d6-4c29-a69c-a53a20e5b0f6 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:26.592703 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a605db9-d1d6-4c29-a69c-a53a20e5b0f6 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:26.601344 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd7f12-a1c1-45ae-8e4d-154601d91507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:26.602334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd7f12-a1c1-45ae-8e4d-154601d91507 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.602621 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd7f12-a1c1-45ae-8e4d-154601d91507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.602905 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd7f12-a1c1-45ae-8e4d-154601d91507 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.603627 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebd7f12-a1c1-45ae-8e4d-154601d91507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:26.606298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebd7f12-a1c1-45ae-8e4d-154601d91507 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:26.621841 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 927/1863] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:26.626245 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd7247-2666-4653-b767-39acd6ba7bd0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:26.637428 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd7247-2666-4653-b767-39acd6ba7bd0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:26.642742 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194bbdf-e098-4778-97cb-3abaf322d493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:26.643375 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194bbdf-e098-4778-97cb-3abaf322d493 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.643659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194bbdf-e098-4778-97cb-3abaf322d493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.643932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194bbdf-e098-4778-97cb-3abaf322d493 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.644637 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194bbdf-e098-4778-97cb-3abaf322d493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:26.646263 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194bbdf-e098-4778-97cb-3abaf322d493 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:26.658603 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 937/1864] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:26.663796 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd7f12-a1c1-45ae-8e4d-154601d91507 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:26.687031 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd7f12-a1c1-45ae-8e4d-154601d91507 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:26.747834 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603e045-9e6e-4b70-96b9-7efc6102c70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:26.748736 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603e045-9e6e-4b70-96b9-7efc6102c70a admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.749209 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603e045-9e6e-4b70-96b9-7efc6102c70a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.749588 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603e045-9e6e-4b70-96b9-7efc6102c70a admin admin] PATH_INFO: `/v3/projects/fdc9a7f7dc9142a7898735f89860ee69/users/498b7468b00b440eb37f001462fb52d4/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.802860 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e603e045-9e6e-4b70-96b9-7efc6102c70a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc9a7f7dc9142a7898735f89860ee69, user_id=498b7468b00b440eb37f001462fb52d4, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:26.809113 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e603e045-9e6e-4b70-96b9-7efc6102c70a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:26.844155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 928/1865] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:26 2022] PUT /identity/v3/projects/fdc9a7f7dc9142a7898735f89860ee69/users/498b7468b00b440eb37f001462fb52d4/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:26.856142 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643d576-bada-4932-ad33-3b157ad1d6c1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:26.861700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643d576-bada-4932-ad33-3b157ad1d6c1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:26.862378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643d576-bada-4932-ad33-3b157ad1d6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:26.925884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0643d576-bada-4932-ad33-3b157ad1d6c1 None None] MFA Rules not processed for user `498b7468b00b440eb37f001462fb52d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:26.927722 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0643d576-bada-4932-ad33-3b157ad1d6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:27.027968 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 938/1866] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:27.041654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603e045-9e6e-4b70-96b9-7efc6102c70a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:27.188489 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603e045-9e6e-4b70-96b9-7efc6102c70a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:27.208619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5ed48-ef9c-440a-8a90-f16362b23f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:27.209972 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5ed48-ef9c-440a-8a90-f16362b23f8e admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:27.210314 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5ed48-ef9c-440a-8a90-f16362b23f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:27.211030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5ed48-ef9c-440a-8a90-f16362b23f8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:27.211807 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5ed48-ef9c-440a-8a90-f16362b23f8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:27.214197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5ed48-ef9c-440a-8a90-f16362b23f8e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:27.252753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 929/1867] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:27 2022] POST /identity/v3/projects => generated 385 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:27.258723 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643d576-bada-4932-ad33-3b157ad1d6c1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:27.276642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643d576-bada-4932-ad33-3b157ad1d6c1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:27.356856 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b8e75-25f7-4247-9599-4bcc5f75b647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:27.357901 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b8e75-25f7-4247-9599-4bcc5f75b647 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:27.358530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b8e75-25f7-4247-9599-4bcc5f75b647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:27.358807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b8e75-25f7-4247-9599-4bcc5f75b647 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:27.359870 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b8e75-25f7-4247-9599-4bcc5f75b647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:27.362714 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b8e75-25f7-4247-9599-4bcc5f75b647 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:27.399991 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 939/1868] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:27 2022] POST /identity/v3/users => generated 299 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:27.401347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5ed48-ef9c-440a-8a90-f16362b23f8e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:27.414475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5ed48-ef9c-440a-8a90-f16362b23f8e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:27.421869 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3aa642-c9ed-4f45-8c46-170ceff68e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:27.421869 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3aa642-c9ed-4f45-8c46-170ceff68e0b admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:27.422358 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3aa642-c9ed-4f45-8c46-170ceff68e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:27.422710 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3aa642-c9ed-4f45-8c46-170ceff68e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:27.423979 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3aa642-c9ed-4f45-8c46-170ceff68e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:27.425765 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3aa642-c9ed-4f45-8c46-170ceff68e0b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:27.448394 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 930/1869] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:56:27.449485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b8e75-25f7-4247-9599-4bcc5f75b647 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:27.492992 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b8e75-25f7-4247-9599-4bcc5f75b647 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:27.499967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b1e3f-23aa-4de8-9263-9b0c889039b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:27.501237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b1e3f-23aa-4de8-9263-9b0c889039b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:27.502300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b1e3f-23aa-4de8-9263-9b0c889039b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:27.502811 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b1e3f-23aa-4de8-9263-9b0c889039b3 admin admin] PATH_INFO: `/v3/projects/350333ebafc745e889bc605b089b0709/users/9fbbcd26505a460585b2f06d883cfa56/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:27.508634 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:27.528943 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b1e3f-23aa-4de8-9263-9b0c889039b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=350333ebafc745e889bc605b089b0709, user_id=9fbbcd26505a460585b2f06d883cfa56, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:27.531695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b1e3f-23aa-4de8-9263-9b0c889039b3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:27.547700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 940/1870] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:27 2022] PUT /identity/v3/projects/350333ebafc745e889bc605b089b0709/users/9fbbcd26505a460585b2f06d883cfa56/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:27.552365 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3aa642-c9ed-4f45-8c46-170ceff68e0b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:27.567620 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3aa642-c9ed-4f45-8c46-170ceff68e0b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:27.603660 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:27.657753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1733bb-cae8-4434-885c-1d07e7fd3532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:27.658598 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1733bb-cae8-4434-885c-1d07e7fd3532 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:27.659077 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1733bb-cae8-4434-885c-1d07e7fd3532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:27.659546 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1733bb-cae8-4434-885c-1d07e7fd3532 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:27.660708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1733bb-cae8-4434-885c-1d07e7fd3532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:27.662351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1733bb-cae8-4434-885c-1d07e7fd3532 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:27.675246 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 931/1871] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:56:27.679540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b1e3f-23aa-4de8-9263-9b0c889039b3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:27.693447 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b1e3f-23aa-4de8-9263-9b0c889039b3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:27.700439 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc8158-2e23-4c66-a535-0a5bbd0efd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:27.701147 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc8158-2e23-4c66-a535-0a5bbd0efd77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:27.701486 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc8158-2e23-4c66-a535-0a5bbd0efd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:27.701873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc8158-2e23-4c66-a535-0a5bbd0efd77 admin admin] PATH_INFO: `/v3/projects/350333ebafc745e889bc605b089b0709/users/9fbbcd26505a460585b2f06d883cfa56/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:27.704025 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dc8158-2e23-4c66-a535-0a5bbd0efd77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=350333ebafc745e889bc605b089b0709, user_id=9fbbcd26505a460585b2f06d883cfa56, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:27.707746 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dc8158-2e23-4c66-a535-0a5bbd0efd77 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:27.732002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 941/1872] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:27 2022] PUT /identity/v3/projects/350333ebafc745e889bc605b089b0709/users/9fbbcd26505a460585b2f06d883cfa56/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:27.743767 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c733f92-6c06-4748-9241-01e49a83f703 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:27.744143 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c733f92-6c06-4748-9241-01e49a83f703 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:27.744432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c733f92-6c06-4748-9241-01e49a83f703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:27.783492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9c733f92-6c06-4748-9241-01e49a83f703 None None] MFA Rules not processed for user `9fbbcd26505a460585b2f06d883cfa56`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:27.785089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9c733f92-6c06-4748-9241-01e49a83f703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:27.924636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 932/1873] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:27 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:27.937971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc8158-2e23-4c66-a535-0a5bbd0efd77 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:27.939738 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-70dc8158-2e23-4c66-a535-0a5bbd0efd77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:28.035933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc8158-2e23-4c66-a535-0a5bbd0efd77 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:28.040369 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9743e356-1e11-4bc8-addb-25301bd684e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:28.041011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743e356-1e11-4bc8-addb-25301bd684e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:28.041295 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743e356-1e11-4bc8-addb-25301bd684e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:28.041567 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743e356-1e11-4bc8-addb-25301bd684e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:28.057183 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9743e356-1e11-4bc8-addb-25301bd684e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:28.058464 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9743e356-1e11-4bc8-addb-25301bd684e6 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:28.074739 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 942/1874] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:28.792132 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c733f92-6c06-4748-9241-01e49a83f703 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:28.793896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9c733f92-6c06-4748-9241-01e49a83f703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:28.806250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c733f92-6c06-4748-9241-01e49a83f703 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:28.809944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb003c70-e2ae-4696-8d2d-8d0e4abec8c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:28.810556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb003c70-e2ae-4696-8d2d-8d0e4abec8c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:28.810833 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb003c70-e2ae-4696-8d2d-8d0e4abec8c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:28.811102 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb003c70-e2ae-4696-8d2d-8d0e4abec8c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:28.876372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb003c70-e2ae-4696-8d2d-8d0e4abec8c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:28.881637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb003c70-e2ae-4696-8d2d-8d0e4abec8c3 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:28.899537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 933/1875] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:31.032164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9743e356-1e11-4bc8-addb-25301bd684e6 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:31.046839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9743e356-1e11-4bc8-addb-25301bd684e6 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:31.053373 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac5ff7-532b-4e49-9c82-535a72676226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:31.054622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac5ff7-532b-4e49-9c82-535a72676226 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:31.055041 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac5ff7-532b-4e49-9c82-535a72676226 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:31.055335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac5ff7-532b-4e49-9c82-535a72676226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:31.057912 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6dac5ff7-532b-4e49-9c82-535a72676226 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:31.194227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac5ff7-532b-4e49-9c82-535a72676226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:31.196065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac5ff7-532b-4e49-9c82-535a72676226 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:31.209565 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 943/1876] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:34.635204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768af52-2670-4ad7-a36d-b99436c52e79 None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:34.635989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768af52-2670-4ad7-a36d-b99436c52e79 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:34.636508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768af52-2670-4ad7-a36d-b99436c52e79 None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:34.639229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 934/1877] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 24 08:56:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:56:34.649445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e3ccb-d785-48cf-a24f-768287fc188c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:34.650293 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e3ccb-d785-48cf-a24f-768287fc188c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:34.650293 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e3ccb-d785-48cf-a24f-768287fc188c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:34.681935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-366e3ccb-d785-48cf-a24f-768287fc188c None None] MFA Rules not processed for user `ca56dd7216fb4946946b2801e144fec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:34.683098 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-366e3ccb-d785-48cf-a24f-768287fc188c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:34.828310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:34.839973 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 944/1878] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 24 08:56:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:56:34.858559 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768af52-2670-4ad7-a36d-b99436c52e79 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:34.860967 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c768af52-2670-4ad7-a36d-b99436c52e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:34.889702 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768af52-2670-4ad7-a36d-b99436c52e79 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:34.896125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c648db-d645-4a17-b372-dc121dc15b2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:34.896882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c648db-d645-4a17-b372-dc121dc15b2f service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:34.897231 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c648db-d645-4a17-b372-dc121dc15b2f service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:34.897589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c648db-d645-4a17-b372-dc121dc15b2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:34.909030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c648db-d645-4a17-b372-dc121dc15b2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:34.911038 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c648db-d645-4a17-b372-dc121dc15b2f service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:34.928769 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 935/1879] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:37.798107 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e3ccb-d785-48cf-a24f-768287fc188c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:37.809001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e3ccb-d785-48cf-a24f-768287fc188c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:37.812903 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c277f4e-d9a0-4fde-bd13-6693bb5f618d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:37.813631 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c277f4e-d9a0-4fde-bd13-6693bb5f618d service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:37.813977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c277f4e-d9a0-4fde-bd13-6693bb5f618d service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:37.814457 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c277f4e-d9a0-4fde-bd13-6693bb5f618d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:37.901149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c277f4e-d9a0-4fde-bd13-6693bb5f618d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:37.902945 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c277f4e-d9a0-4fde-bd13-6693bb5f618d service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:37.915951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 945/1880] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:37 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:44.921347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=86 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:46.717427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c648db-d645-4a17-b372-dc121dc15b2f service nova] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:46.923423 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c648db-d645-4a17-b372-dc121dc15b2f service nova] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:46.941083 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5d373-5e45-4e73-99fe-2161199c1eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:46.941768 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5d373-5e45-4e73-99fe-2161199c1eb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:46.942058 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5d373-5e45-4e73-99fe-2161199c1eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:46.942326 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5d373-5e45-4e73-99fe-2161199c1eb7 admin admin] PATH_INFO: `/v3/users/df03267bb8c646d1b2ffc1706e45eef8` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:46.943638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5d373-5e45-4e73-99fe-2161199c1eb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df03267bb8c646d1b2ffc1706e45eef8)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:46.945711 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5d373-5e45-4e73-99fe-2161199c1eb7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:47.052039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fed5d373-5e45-4e73-99fe-2161199c1eb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df03267bb8c646d1b2ffc1706e45eef8'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:47.074648 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fed5d373-5e45-4e73-99fe-2161199c1eb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df03267bb8c646d1b2ffc1706e45eef8'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:47.111670 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fed5d373-5e45-4e73-99fe-2161199c1eb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df03267bb8c646d1b2ffc1706e45eef8'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:47.135380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fed5d373-5e45-4e73-99fe-2161199c1eb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df03267bb8c646d1b2ffc1706e45eef8'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:47.137575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fed5d373-5e45-4e73-99fe-2161199c1eb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df03267bb8c646d1b2ffc1706e45eef8'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:47.147879 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 936/1881] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:46 2022] DELETE /identity/v3/users/df03267bb8c646d1b2ffc1706e45eef8 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:47.152742 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431be51e-639c-460e-9e5d-38ea18ccfcaa None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:47.153756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431be51e-639c-460e-9e5d-38ea18ccfcaa None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:47.154037 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431be51e-639c-460e-9e5d-38ea18ccfcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:47.177111 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-431be51e-639c-460e-9e5d-38ea18ccfcaa None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:47.178444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-431be51e-639c-460e-9e5d-38ea18ccfcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:47.263681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 946/1882] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:47.277821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5d373-5e45-4e73-99fe-2161199c1eb7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:47.280136 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fed5d373-5e45-4e73-99fe-2161199c1eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:47.380245 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5d373-5e45-4e73-99fe-2161199c1eb7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:47.384670 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2141ee-ddd8-4405-939c-76526d165239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:47.385636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2141ee-ddd8-4405-939c-76526d165239 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:47.385994 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2141ee-ddd8-4405-939c-76526d165239 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:47.386374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2141ee-ddd8-4405-939c-76526d165239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:47.399706 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2141ee-ddd8-4405-939c-76526d165239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:47.400915 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2141ee-ddd8-4405-939c-76526d165239 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:47.412840 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 937/1883] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:56:47.816643 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431be51e-639c-460e-9e5d-38ea18ccfcaa None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:47.818452 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-431be51e-639c-460e-9e5d-38ea18ccfcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:47.833466 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431be51e-639c-460e-9e5d-38ea18ccfcaa None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:47.861557 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ef08e-958f-48fa-99fb-6fc9bf8dda8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:47.862219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ef08e-958f-48fa-99fb-6fc9bf8dda8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:47.862504 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ef08e-958f-48fa-99fb-6fc9bf8dda8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:47.862777 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ef08e-958f-48fa-99fb-6fc9bf8dda8d admin admin] PATH_INFO: `/v3/projects/3dfa220132114b60b3607e8b4687bda5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:47.864134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ef08e-958f-48fa-99fb-6fc9bf8dda8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dfa220132114b60b3607e8b4687bda5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:47.866413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ef08e-958f-48fa-99fb-6fc9bf8dda8d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:47.925374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-658ef08e-958f-48fa-99fb-6fc9bf8dda8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dfa220132114b60b3607e8b4687bda5', 'request_id': 'req-658ef08e-958f-48fa-99fb-6fc9bf8dda8d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:47.932214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-658ef08e-958f-48fa-99fb-6fc9bf8dda8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dfa220132114b60b3607e8b4687bda5', 'request_id': 'req-658ef08e-958f-48fa-99fb-6fc9bf8dda8d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:47.942806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-658ef08e-958f-48fa-99fb-6fc9bf8dda8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dfa220132114b60b3607e8b4687bda5', 'request_id': 'req-658ef08e-958f-48fa-99fb-6fc9bf8dda8d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:47.959296 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-658ef08e-958f-48fa-99fb-6fc9bf8dda8d admin admin] The token cache is being invalidate because project 3dfa220132114b60b3607e8b4687bda5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:47.959817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-658ef08e-958f-48fa-99fb-6fc9bf8dda8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:47.961033 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-658ef08e-958f-48fa-99fb-6fc9bf8dda8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:47.962827 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 947/1884] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:47 2022] DELETE /identity/v3/projects/3dfa220132114b60b3607e8b4687bda5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:47.967322 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2141ee-ddd8-4405-939c-76526d165239 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:47.978348 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2141ee-ddd8-4405-939c-76526d165239 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:48.000753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:48.059785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25082e-9a99-4800-8805-f46064ebdf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:48.059785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25082e-9a99-4800-8805-f46064ebdf17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:48.059785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25082e-9a99-4800-8805-f46064ebdf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:48.059785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25082e-9a99-4800-8805-f46064ebdf17 admin admin] PATH_INFO: `/v3/users/498b7468b00b440eb37f001462fb52d4` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:48.061723 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da25082e-9a99-4800-8805-f46064ebdf17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=498b7468b00b440eb37f001462fb52d4)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:48.082161 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da25082e-9a99-4800-8805-f46064ebdf17 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:48.161939 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-da25082e-9a99-4800-8805-f46064ebdf17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '498b7468b00b440eb37f001462fb52d4'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:48.173409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-da25082e-9a99-4800-8805-f46064ebdf17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '498b7468b00b440eb37f001462fb52d4'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:48.193481 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-da25082e-9a99-4800-8805-f46064ebdf17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '498b7468b00b440eb37f001462fb52d4'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:48.203244 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-da25082e-9a99-4800-8805-f46064ebdf17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '498b7468b00b440eb37f001462fb52d4'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:48.204051 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-da25082e-9a99-4800-8805-f46064ebdf17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '498b7468b00b440eb37f001462fb52d4'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:48.211596 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 938/1885] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:47 2022] DELETE /identity/v3/users/498b7468b00b440eb37f001462fb52d4 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:48.544416 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ef08e-958f-48fa-99fb-6fc9bf8dda8d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:48.546733 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-658ef08e-958f-48fa-99fb-6fc9bf8dda8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:48.604341 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ef08e-958f-48fa-99fb-6fc9bf8dda8d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:48.641543 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:48.735948 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143d168-6c51-4850-9fe0-74701ec32d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:48.735948 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143d168-6c51-4850-9fe0-74701ec32d0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:48.735948 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143d168-6c51-4850-9fe0-74701ec32d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:48.735948 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143d168-6c51-4850-9fe0-74701ec32d0d admin admin] PATH_INFO: `/v3/projects/fdc9a7f7dc9142a7898735f89860ee69` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:48.736727 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143d168-6c51-4850-9fe0-74701ec32d0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdc9a7f7dc9142a7898735f89860ee69)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:48.742400 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143d168-6c51-4850-9fe0-74701ec32d0d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:48.810426 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9143d168-6c51-4850-9fe0-74701ec32d0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdc9a7f7dc9142a7898735f89860ee69', 'request_id': 'req-9143d168-6c51-4850-9fe0-74701ec32d0d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:48.822780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9143d168-6c51-4850-9fe0-74701ec32d0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdc9a7f7dc9142a7898735f89860ee69', 'request_id': 'req-9143d168-6c51-4850-9fe0-74701ec32d0d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:48.844500 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9143d168-6c51-4850-9fe0-74701ec32d0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdc9a7f7dc9142a7898735f89860ee69', 'request_id': 'req-9143d168-6c51-4850-9fe0-74701ec32d0d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:48.859885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9143d168-6c51-4850-9fe0-74701ec32d0d admin admin] The token cache is being invalidate because project fdc9a7f7dc9142a7898735f89860ee69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:48.860230 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9143d168-6c51-4850-9fe0-74701ec32d0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:48.861104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9143d168-6c51-4850-9fe0-74701ec32d0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:48.863667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 948/1886] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:48 2022] DELETE /identity/v3/projects/fdc9a7f7dc9142a7898735f89860ee69 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:48.870203 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25082e-9a99-4800-8805-f46064ebdf17 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:48.871933 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-da25082e-9a99-4800-8805-f46064ebdf17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:48.888658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25082e-9a99-4800-8805-f46064ebdf17 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:48.981884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8150b-8850-4521-9048-65180f6a3e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:48.983301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8150b-8850-4521-9048-65180f6a3e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:48.984125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8150b-8850-4521-9048-65180f6a3e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:48.984716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8150b-8850-4521-9048-65180f6a3e13 admin admin] PATH_INFO: `/v3/users/9fbbcd26505a460585b2f06d883cfa56` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:48.987180 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8150b-8850-4521-9048-65180f6a3e13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fbbcd26505a460585b2f06d883cfa56)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:48.991591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8150b-8850-4521-9048-65180f6a3e13 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:49.064529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:49.078514 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d0b8150b-8850-4521-9048-65180f6a3e13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fbbcd26505a460585b2f06d883cfa56'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.092011 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d0b8150b-8850-4521-9048-65180f6a3e13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fbbcd26505a460585b2f06d883cfa56'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.120507 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d0b8150b-8850-4521-9048-65180f6a3e13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fbbcd26505a460585b2f06d883cfa56'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.141729 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d0b8150b-8850-4521-9048-65180f6a3e13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fbbcd26505a460585b2f06d883cfa56'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.142656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d0b8150b-8850-4521-9048-65180f6a3e13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fbbcd26505a460585b2f06d883cfa56'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.156157 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 939/1887] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:56:48 2022] DELETE /identity/v3/users/9fbbcd26505a460585b2f06d883cfa56 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:49.707801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143d168-6c51-4850-9fe0-74701ec32d0d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:49.710203 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9143d168-6c51-4850-9fe0-74701ec32d0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:49.728663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143d168-6c51-4850-9fe0-74701ec32d0d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:49.806890 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e2a9f-895b-4139-b663-1e6a9a568542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:49.807453 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e2a9f-895b-4139-b663-1e6a9a568542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:49.807737 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e2a9f-895b-4139-b663-1e6a9a568542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:49.808014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e2a9f-895b-4139-b663-1e6a9a568542 admin admin] PATH_INFO: `/v3/projects/350333ebafc745e889bc605b089b0709` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:49.809322 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e2a9f-895b-4139-b663-1e6a9a568542 admin admin] RBAC: Authorizing `identity:delete_project(project_id=350333ebafc745e889bc605b089b0709)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:49.811436 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e2a9f-895b-4139-b663-1e6a9a568542 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:49.886499 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cb6e2a9f-895b-4139-b663-1e6a9a568542 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '350333ebafc745e889bc605b089b0709', 'request_id': 'req-cb6e2a9f-895b-4139-b663-1e6a9a568542'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.891809 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:49.896566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cb6e2a9f-895b-4139-b663-1e6a9a568542 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '350333ebafc745e889bc605b089b0709', 'request_id': 'req-cb6e2a9f-895b-4139-b663-1e6a9a568542'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.911339 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cb6e2a9f-895b-4139-b663-1e6a9a568542 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '350333ebafc745e889bc605b089b0709', 'request_id': 'req-cb6e2a9f-895b-4139-b663-1e6a9a568542'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.930809 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cb6e2a9f-895b-4139-b663-1e6a9a568542 admin admin] The token cache is being invalidate because project 350333ebafc745e889bc605b089b0709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:56:49.931300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cb6e2a9f-895b-4139-b663-1e6a9a568542 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.932024 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cb6e2a9f-895b-4139-b663-1e6a9a568542 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:56:49.933677 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 949/1888] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:56:49 2022] DELETE /identity/v3/projects/350333ebafc745e889bc605b089b0709 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:56:49.940000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743a832-3cd9-4c58-93bd-0535248b8eb9 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:49.941234 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743a832-3cd9-4c58-93bd-0535248b8eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:49.941629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743a832-3cd9-4c58-93bd-0535248b8eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:49.964841 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-3743a832-3cd9-4c58-93bd-0535248b8eb9 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:49.966107 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3743a832-3cd9-4c58-93bd-0535248b8eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:50.048024 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 940/1889] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:50.058126 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360522d6-ffa5-4e01-977c-4b5487885a03 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:50.058360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360522d6-ffa5-4e01-977c-4b5487885a03 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:50.058573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360522d6-ffa5-4e01-977c-4b5487885a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:50.105487 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-360522d6-ffa5-4e01-977c-4b5487885a03 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:50.106771 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-360522d6-ffa5-4e01-977c-4b5487885a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:50.118972 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 950/1890] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:50.125264 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743a832-3cd9-4c58-93bd-0535248b8eb9 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:50.147845 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743a832-3cd9-4c58-93bd-0535248b8eb9 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:50.154405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78c5c8-9c0c-4e97-ae96-8b05bac2b09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:50.155193 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78c5c8-9c0c-4e97-ae96-8b05bac2b09e admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:50.155562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78c5c8-9c0c-4e97-ae96-8b05bac2b09e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:50.156248 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78c5c8-9c0c-4e97-ae96-8b05bac2b09e admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:50.157071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78c5c8-9c0c-4e97-ae96-8b05bac2b09e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:56:50.157597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78c5c8-9c0c-4e97-ae96-8b05bac2b09e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:50.159533 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78c5c8-9c0c-4e97-ae96-8b05bac2b09e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:50.187840 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 941/1891] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:56:50.191361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360522d6-ffa5-4e01-977c-4b5487885a03 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:50.212615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360522d6-ffa5-4e01-977c-4b5487885a03 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:50.217562 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9338aba-4497-407f-9d12-f29e56a2da04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:50.218194 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9338aba-4497-407f-9d12-f29e56a2da04 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:50.218480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9338aba-4497-407f-9d12-f29e56a2da04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:50.218758 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9338aba-4497-407f-9d12-f29e56a2da04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:50.219410 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9338aba-4497-407f-9d12-f29e56a2da04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:50.221521 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9338aba-4497-407f-9d12-f29e56a2da04 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:50.258535 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 951/1892] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:56:50 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:50.263123 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78c5c8-9c0c-4e97-ae96-8b05bac2b09e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:50.278954 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78c5c8-9c0c-4e97-ae96-8b05bac2b09e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:50.375755 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea39fe6-f421-4c7c-b0b6-63585bf883e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:50.376722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea39fe6-f421-4c7c-b0b6-63585bf883e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:50.377231 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea39fe6-f421-4c7c-b0b6-63585bf883e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:50.377889 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea39fe6-f421-4c7c-b0b6-63585bf883e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:50.379723 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea39fe6-f421-4c7c-b0b6-63585bf883e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:50.382994 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea39fe6-f421-4c7c-b0b6-63585bf883e0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:50.416204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 942/1893] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:56:50 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:56:50.417817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9338aba-4497-407f-9d12-f29e56a2da04 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:50.433929 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9338aba-4497-407f-9d12-f29e56a2da04 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:50.448302 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc467a9-03ce-4b28-8a78-2aa41436e7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:50.448302 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc467a9-03ce-4b28-8a78-2aa41436e7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:50.448302 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc467a9-03ce-4b28-8a78-2aa41436e7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:50.448900 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc467a9-03ce-4b28-8a78-2aa41436e7f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:50.449286 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc467a9-03ce-4b28-8a78-2aa41436e7f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:50.451962 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc467a9-03ce-4b28-8a78-2aa41436e7f7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:50.463614 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 952/1894] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:56:50.468440 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea39fe6-f421-4c7c-b0b6-63585bf883e0 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:50.482547 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea39fe6-f421-4c7c-b0b6-63585bf883e0 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:50.489965 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda2b00-1860-4490-a82e-2a1730a4f000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:50.490980 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda2b00-1860-4490-a82e-2a1730a4f000 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:50.491269 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda2b00-1860-4490-a82e-2a1730a4f000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:50.491541 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda2b00-1860-4490-a82e-2a1730a4f000 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:50.492235 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda2b00-1860-4490-a82e-2a1730a4f000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:50.493933 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda2b00-1860-4490-a82e-2a1730a4f000 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:50.511874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 943/1895] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:56:50.513503 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc467a9-03ce-4b28-8a78-2aa41436e7f7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:50.527548 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc467a9-03ce-4b28-8a78-2aa41436e7f7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:50.532636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d4895-15a9-4680-bdd9-79d10d5b93d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:50.533324 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d4895-15a9-4680-bdd9-79d10d5b93d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:50.533605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d4895-15a9-4680-bdd9-79d10d5b93d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:50.533880 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d4895-15a9-4680-bdd9-79d10d5b93d7 admin admin] PATH_INFO: `/v3/projects/6fc955bcd7af4bc09cfdc2f134ee827d/users/7b52e94c20d0419ea8e597bf85b91ed4/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:50.574173 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d4895-15a9-4680-bdd9-79d10d5b93d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fc955bcd7af4bc09cfdc2f134ee827d, user_id=7b52e94c20d0419ea8e597bf85b91ed4, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:50.577922 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d4895-15a9-4680-bdd9-79d10d5b93d7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:50.601183 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 953/1896] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:56:50 2022] PUT /identity/v3/projects/6fc955bcd7af4bc09cfdc2f134ee827d/users/7b52e94c20d0419ea8e597bf85b91ed4/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:56:50.614116 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd57490-5c76-4d06-99db-ac725f13c1b5 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:50.614398 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd57490-5c76-4d06-99db-ac725f13c1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:50.614669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd57490-5c76-4d06-99db-ac725f13c1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:50.646789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9dd57490-5c76-4d06-99db-ac725f13c1b5 None None] MFA Rules not processed for user `7b52e94c20d0419ea8e597bf85b91ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:56:50.648337 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9dd57490-5c76-4d06-99db-ac725f13c1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:50.675695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:56:50.752050 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 944/1897] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:56:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:56:50.765742 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d4895-15a9-4680-bdd9-79d10d5b93d7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:56:50.767598 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-936d4895-15a9-4680-bdd9-79d10d5b93d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:56:50.839113 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d4895-15a9-4680-bdd9-79d10d5b93d7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:56:50.843371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79085e-d8ac-4e56-85b2-a38e1acd7446 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:56:50.844013 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79085e-d8ac-4e56-85b2-a38e1acd7446 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:56:50.844324 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79085e-d8ac-4e56-85b2-a38e1acd7446 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:56:50.844624 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79085e-d8ac-4e56-85b2-a38e1acd7446 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:56:50.855472 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79085e-d8ac-4e56-85b2-a38e1acd7446 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:56:50.856859 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79085e-d8ac-4e56-85b2-a38e1acd7446 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:56:50.867812 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 954/1898] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:56:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:00.747131 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=20 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:00.862261 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=2,keystone:SELECT=58,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:14.503726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd57490-5c76-4d06-99db-ac725f13c1b5 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:14.506033 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9dd57490-5c76-4d06-99db-ac725f13c1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:14.580650 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd57490-5c76-4d06-99db-ac725f13c1b5 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:14.585995 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf1011-5638-402e-9686-71ae07f18a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:14.586644 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf1011-5638-402e-9686-71ae07f18a72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:14.586926 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf1011-5638-402e-9686-71ae07f18a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:14.587202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf1011-5638-402e-9686-71ae07f18a72 admin admin] PATH_INFO: `/v3/users/91ec6d1fef304f608af68eaf67f1dbff` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:14.588370 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf1011-5638-402e-9686-71ae07f18a72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91ec6d1fef304f608af68eaf67f1dbff)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:14.590312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf1011-5638-402e-9686-71ae07f18a72 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:14.643036 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-45bf1011-5638-402e-9686-71ae07f18a72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91ec6d1fef304f608af68eaf67f1dbff'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:14.651659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-45bf1011-5638-402e-9686-71ae07f18a72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91ec6d1fef304f608af68eaf67f1dbff'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:14.666651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-45bf1011-5638-402e-9686-71ae07f18a72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91ec6d1fef304f608af68eaf67f1dbff'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:14.674314 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-45bf1011-5638-402e-9686-71ae07f18a72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91ec6d1fef304f608af68eaf67f1dbff'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:14.675105 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-45bf1011-5638-402e-9686-71ae07f18a72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91ec6d1fef304f608af68eaf67f1dbff'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:14.686384 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 945/1899] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:14 2022] DELETE /identity/v3/users/91ec6d1fef304f608af68eaf67f1dbff => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:14.690959 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186c8e1-cace-4c38-bed8-ece019170e4c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:14.691453 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186c8e1-cace-4c38-bed8-ece019170e4c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:14.692059 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186c8e1-cace-4c38-bed8-ece019170e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:14.719422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4186c8e1-cace-4c38-bed8-ece019170e4c None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:14.720871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4186c8e1-cace-4c38-bed8-ece019170e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:14.796287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 955/1900] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:14.806112 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf1011-5638-402e-9686-71ae07f18a72 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:14.807752 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-45bf1011-5638-402e-9686-71ae07f18a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:14.877284 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf1011-5638-402e-9686-71ae07f18a72 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:14.884374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbc5d5-f5b8-4e96-ab95-5c69495994d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:14.885068 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbc5d5-f5b8-4e96-ab95-5c69495994d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:14.885353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbc5d5-f5b8-4e96-ab95-5c69495994d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:14.885623 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbc5d5-f5b8-4e96-ab95-5c69495994d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:14.896047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbc5d5-f5b8-4e96-ab95-5c69495994d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:14.897536 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbc5d5-f5b8-4e96-ab95-5c69495994d3 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:14.908273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 946/1901] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:15.314783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4186c8e1-cace-4c38-bed8-ece019170e4c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:15.317564 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4186c8e1-cace-4c38-bed8-ece019170e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:15.342780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4186c8e1-cace-4c38-bed8-ece019170e4c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:15.352054 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c9328-d71d-4355-8085-35e6f8d1b588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:15.353177 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c9328-d71d-4355-8085-35e6f8d1b588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:15.353681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c9328-d71d-4355-8085-35e6f8d1b588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:15.354878 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c9328-d71d-4355-8085-35e6f8d1b588 admin admin] PATH_INFO: `/v3/projects/532d1831dfbb43dfba6f3f20ebc8153a` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:15.356937 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1c9328-d71d-4355-8085-35e6f8d1b588 admin admin] RBAC: Authorizing `identity:delete_project(project_id=532d1831dfbb43dfba6f3f20ebc8153a)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:15.360785 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1c9328-d71d-4355-8085-35e6f8d1b588 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:15.424372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6c1c9328-d71d-4355-8085-35e6f8d1b588 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '532d1831dfbb43dfba6f3f20ebc8153a', 'request_id': 'req-6c1c9328-d71d-4355-8085-35e6f8d1b588'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.435639 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6c1c9328-d71d-4355-8085-35e6f8d1b588 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '532d1831dfbb43dfba6f3f20ebc8153a', 'request_id': 'req-6c1c9328-d71d-4355-8085-35e6f8d1b588'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.455204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6c1c9328-d71d-4355-8085-35e6f8d1b588 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '532d1831dfbb43dfba6f3f20ebc8153a', 'request_id': 'req-6c1c9328-d71d-4355-8085-35e6f8d1b588'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.472486 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6c1c9328-d71d-4355-8085-35e6f8d1b588 admin admin] The token cache is being invalidate because project 532d1831dfbb43dfba6f3f20ebc8153a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:15.473011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6c1c9328-d71d-4355-8085-35e6f8d1b588 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.473934 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6c1c9328-d71d-4355-8085-35e6f8d1b588 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.476868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 956/1902] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:15 2022] DELETE /identity/v3/projects/532d1831dfbb43dfba6f3f20ebc8153a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:15.481000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbc5d5-f5b8-4e96-ab95-5c69495994d3 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:15.491297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbc5d5-f5b8-4e96-ab95-5c69495994d3 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:15.511567 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:15.555021 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a28cda-3073-4c09-ba1a-394da09c52c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:15.555626 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a28cda-3073-4c09-ba1a-394da09c52c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:15.555903 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a28cda-3073-4c09-ba1a-394da09c52c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:15.556232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a28cda-3073-4c09-ba1a-394da09c52c9 admin admin] PATH_INFO: `/v3/users/5461439bc88840619ef97302ed48c7a6` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:15.557354 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a28cda-3073-4c09-ba1a-394da09c52c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5461439bc88840619ef97302ed48c7a6)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:15.559272 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a28cda-3073-4c09-ba1a-394da09c52c9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:15.608734 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-94a28cda-3073-4c09-ba1a-394da09c52c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5461439bc88840619ef97302ed48c7a6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.618757 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-94a28cda-3073-4c09-ba1a-394da09c52c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5461439bc88840619ef97302ed48c7a6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.631678 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-94a28cda-3073-4c09-ba1a-394da09c52c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5461439bc88840619ef97302ed48c7a6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.638504 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-94a28cda-3073-4c09-ba1a-394da09c52c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5461439bc88840619ef97302ed48c7a6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.639209 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-94a28cda-3073-4c09-ba1a-394da09c52c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5461439bc88840619ef97302ed48c7a6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:15.645733 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 947/1903] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:15 2022] DELETE /identity/v3/users/5461439bc88840619ef97302ed48c7a6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:15.963233 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c9328-d71d-4355-8085-35e6f8d1b588 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:15.964798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6c1c9328-d71d-4355-8085-35e6f8d1b588 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:15.983506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c9328-d71d-4355-8085-35e6f8d1b588 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:16.074885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f925b-9b12-4690-9dd5-0c7bb7518017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:16.076370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f925b-9b12-4690-9dd5-0c7bb7518017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:16.076880 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f925b-9b12-4690-9dd5-0c7bb7518017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:16.077397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f925b-9b12-4690-9dd5-0c7bb7518017 admin admin] PATH_INFO: `/v3/projects/8f239f1a387e48ee9ddaa381e3f4882b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:16.078992 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f925b-9b12-4690-9dd5-0c7bb7518017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f239f1a387e48ee9ddaa381e3f4882b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:16.081642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f925b-9b12-4690-9dd5-0c7bb7518017 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:16.125425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:16.159460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-232f925b-9b12-4690-9dd5-0c7bb7518017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f239f1a387e48ee9ddaa381e3f4882b', 'request_id': 'req-232f925b-9b12-4690-9dd5-0c7bb7518017'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:16.169377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-232f925b-9b12-4690-9dd5-0c7bb7518017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f239f1a387e48ee9ddaa381e3f4882b', 'request_id': 'req-232f925b-9b12-4690-9dd5-0c7bb7518017'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:16.181090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-232f925b-9b12-4690-9dd5-0c7bb7518017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f239f1a387e48ee9ddaa381e3f4882b', 'request_id': 'req-232f925b-9b12-4690-9dd5-0c7bb7518017'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:16.201091 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-232f925b-9b12-4690-9dd5-0c7bb7518017 admin admin] The token cache is being invalidate because project 8f239f1a387e48ee9ddaa381e3f4882b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:16.201387 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-232f925b-9b12-4690-9dd5-0c7bb7518017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:16.202079 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-232f925b-9b12-4690-9dd5-0c7bb7518017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:16.204295 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 957/1904] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:15 2022] DELETE /identity/v3/projects/8f239f1a387e48ee9ddaa381e3f4882b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:16.211195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e2a7b-fc9f-49bc-bd65-1c4c9e8e6530 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:16.211703 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e2a7b-fc9f-49bc-bd65-1c4c9e8e6530 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:16.212236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e2a7b-fc9f-49bc-bd65-1c4c9e8e6530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:16.247634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d00e2a7b-fc9f-49bc-bd65-1c4c9e8e6530 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:16.249890 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d00e2a7b-fc9f-49bc-bd65-1c4c9e8e6530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:16.373975 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 948/1905] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:16.383786 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991078a0-0bfa-46fc-acc6-dd2686c95453 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:16.384070 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991078a0-0bfa-46fc-acc6-dd2686c95453 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:16.384484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991078a0-0bfa-46fc-acc6-dd2686c95453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:16.421703 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-991078a0-0bfa-46fc-acc6-dd2686c95453 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:16.422998 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-991078a0-0bfa-46fc-acc6-dd2686c95453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:16.435334 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 958/1906] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:16.438296 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e2a7b-fc9f-49bc-bd65-1c4c9e8e6530 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:16.449754 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e2a7b-fc9f-49bc-bd65-1c4c9e8e6530 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:16.454901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df1e23-9042-4be0-a1a7-ebc68412d6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:16.455542 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df1e23-9042-4be0-a1a7-ebc68412d6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:16.455819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df1e23-9042-4be0-a1a7-ebc68412d6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:16.456140 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df1e23-9042-4be0-a1a7-ebc68412d6c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:16.456876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df1e23-9042-4be0-a1a7-ebc68412d6c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:16.457198 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df1e23-9042-4be0-a1a7-ebc68412d6c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:16.459182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df1e23-9042-4be0-a1a7-ebc68412d6c1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:16.473117 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 949/1907] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:57:16.478278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991078a0-0bfa-46fc-acc6-dd2686c95453 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:16.489875 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991078a0-0bfa-46fc-acc6-dd2686c95453 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:16.495787 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c66ed-68f4-44f0-a428-4241ebb61c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:16.496448 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c66ed-68f4-44f0-a428-4241ebb61c4c admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:16.496730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c66ed-68f4-44f0-a428-4241ebb61c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:16.497004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c66ed-68f4-44f0-a428-4241ebb61c4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:16.497654 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c66ed-68f4-44f0-a428-4241ebb61c4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:16.499592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c66ed-68f4-44f0-a428-4241ebb61c4c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:16.532249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 959/1908] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:16 2022] POST /identity/v3/projects => generated 399 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:16.537471 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df1e23-9042-4be0-a1a7-ebc68412d6c1 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:16.549460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df1e23-9042-4be0-a1a7-ebc68412d6c1 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:16.590420 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:16.652201 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122479ff-f28a-454f-b4fd-4bd5ef484d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:16.653113 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122479ff-f28a-454f-b4fd-4bd5ef484d34 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:16.653404 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122479ff-f28a-454f-b4fd-4bd5ef484d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:16.653687 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122479ff-f28a-454f-b4fd-4bd5ef484d34 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:16.654300 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122479ff-f28a-454f-b4fd-4bd5ef484d34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:16.656266 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122479ff-f28a-454f-b4fd-4bd5ef484d34 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:16.683794 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 950/1909] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:16 2022] POST /identity/v3/users => generated 300 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:16.685701 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c66ed-68f4-44f0-a428-4241ebb61c4c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:16.705437 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c66ed-68f4-44f0-a428-4241ebb61c4c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:16.710520 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e185327-3529-437b-9959-41e951b3ea2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:16.711480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e185327-3529-437b-9959-41e951b3ea2a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:16.711757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e185327-3529-437b-9959-41e951b3ea2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:16.712034 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e185327-3529-437b-9959-41e951b3ea2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:16.712723 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e185327-3529-437b-9959-41e951b3ea2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:16.714471 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e185327-3529-437b-9959-41e951b3ea2a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:16.732582 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 960/1910] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:16.733843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122479ff-f28a-454f-b4fd-4bd5ef484d34 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:16.746265 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122479ff-f28a-454f-b4fd-4bd5ef484d34 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:16.751545 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c167ff-3937-4665-ae72-145e71aee7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:16.752221 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c167ff-3937-4665-ae72-145e71aee7f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:16.752802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c167ff-3937-4665-ae72-145e71aee7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:16.753085 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c167ff-3937-4665-ae72-145e71aee7f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:16.753745 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c167ff-3937-4665-ae72-145e71aee7f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:16.756233 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c167ff-3937-4665-ae72-145e71aee7f3 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:16.771763 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 951/1911] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:16.773105 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e185327-3529-437b-9959-41e951b3ea2a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:16.783893 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e185327-3529-437b-9959-41e951b3ea2a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:16.790550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bf1e5-9387-4d97-9a6a-82a8cde90ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:16.791335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bf1e5-9387-4d97-9a6a-82a8cde90ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:16.791631 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bf1e5-9387-4d97-9a6a-82a8cde90ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:16.791907 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bf1e5-9387-4d97-9a6a-82a8cde90ba8 admin admin] PATH_INFO: `/v3/projects/b639e83157e8404e9b1fd1ee9a8f591b/users/4d78fca1ff3a4dbeb226e3b1c5869a8f/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:16.811345 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bf1e5-9387-4d97-9a6a-82a8cde90ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b639e83157e8404e9b1fd1ee9a8f591b, user_id=4d78fca1ff3a4dbeb226e3b1c5869a8f, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:16.814118 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bf1e5-9387-4d97-9a6a-82a8cde90ba8 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:16.829044 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 961/1912] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:16 2022] PUT /identity/v3/projects/b639e83157e8404e9b1fd1ee9a8f591b/users/4d78fca1ff3a4dbeb226e3b1c5869a8f/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:16.841883 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7831176-9e64-453b-a9c2-b8071e1a2cd8 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:16.842264 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7831176-9e64-453b-a9c2-b8071e1a2cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:16.842539 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7831176-9e64-453b-a9c2-b8071e1a2cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:16.875605 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b7831176-9e64-453b-a9c2-b8071e1a2cd8 None None] MFA Rules not processed for user `4d78fca1ff3a4dbeb226e3b1c5869a8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:16.877203 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b7831176-9e64-453b-a9c2-b8071e1a2cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:16.969452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 952/1913] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:16 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:16.975619 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bf1e5-9387-4d97-9a6a-82a8cde90ba8 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:17.043445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bf1e5-9387-4d97-9a6a-82a8cde90ba8 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:17.048512 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a363bea-1641-4a5e-8875-8baccea3a4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:17.049127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a363bea-1641-4a5e-8875-8baccea3a4ed admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.049409 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a363bea-1641-4a5e-8875-8baccea3a4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.049678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a363bea-1641-4a5e-8875-8baccea3a4ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:17.050322 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a363bea-1641-4a5e-8875-8baccea3a4ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:17.052463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a363bea-1641-4a5e-8875-8baccea3a4ed admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:17.088504 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 962/1914] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:16 2022] POST /identity/v3/projects => generated 401 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:17.093438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7831176-9e64-453b-a9c2-b8071e1a2cd8 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:17.105515 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7831176-9e64-453b-a9c2-b8071e1a2cd8 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:17.174701 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97e547-4571-4a42-9940-0db9666e93bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:17.175532 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97e547-4571-4a42-9940-0db9666e93bb admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.175856 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97e547-4571-4a42-9940-0db9666e93bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.176860 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97e547-4571-4a42-9940-0db9666e93bb admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:17.177977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97e547-4571-4a42-9940-0db9666e93bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:17.180009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97e547-4571-4a42-9940-0db9666e93bb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:17.205272 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 953/1915] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:17 2022] POST /identity/v3/users => generated 307 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:17.209623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a363bea-1641-4a5e-8875-8baccea3a4ed admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:17.229198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a363bea-1641-4a5e-8875-8baccea3a4ed admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:17.234308 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8a370-3ecb-4153-a74d-84590c9dc86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:17.234939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8a370-3ecb-4153-a74d-84590c9dc86a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.235219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8a370-3ecb-4153-a74d-84590c9dc86a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.235490 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8a370-3ecb-4153-a74d-84590c9dc86a admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:17.236173 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8a370-3ecb-4153-a74d-84590c9dc86a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:17.237810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8a370-3ecb-4153-a74d-84590c9dc86a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:17.250488 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 963/1916] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:17.255787 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97e547-4571-4a42-9940-0db9666e93bb admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:17.266878 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97e547-4571-4a42-9940-0db9666e93bb admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:17.271919 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2ec5d-d1a7-420a-ac40-9369005af5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:17.272641 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2ec5d-d1a7-420a-ac40-9369005af5d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.272925 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2ec5d-d1a7-420a-ac40-9369005af5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.273229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2ec5d-d1a7-420a-ac40-9369005af5d5 admin admin] PATH_INFO: `/v3/projects/fdfe32930eb64a149f4dd6af361a9201/users/07d12ff9a6254fb9a32e1788f9920796/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:17.290987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2ec5d-d1a7-420a-ac40-9369005af5d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdfe32930eb64a149f4dd6af361a9201, user_id=07d12ff9a6254fb9a32e1788f9920796, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:17.293794 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2ec5d-d1a7-420a-ac40-9369005af5d5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:17.311422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 954/1917] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:17 2022] PUT /identity/v3/projects/fdfe32930eb64a149f4dd6af361a9201/users/07d12ff9a6254fb9a32e1788f9920796/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:17.317855 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8a370-3ecb-4153-a74d-84590c9dc86a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:17.328616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8a370-3ecb-4153-a74d-84590c9dc86a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:17.369673 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:17.454027 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed1104-e96b-42d6-8970-590bfc91aa2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:17.454874 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed1104-e96b-42d6-8970-590bfc91aa2e admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.455214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed1104-e96b-42d6-8970-590bfc91aa2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.455556 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed1104-e96b-42d6-8970-590bfc91aa2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:17.456350 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed1104-e96b-42d6-8970-590bfc91aa2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:17.459014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed1104-e96b-42d6-8970-590bfc91aa2e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:17.476136 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 964/1918] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:17.482984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2ec5d-d1a7-420a-ac40-9369005af5d5 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:17.489600 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:17.497978 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2ec5d-d1a7-420a-ac40-9369005af5d5 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:17.507658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ab4ae-7548-4ffd-a30d-262a4780c8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:17.508968 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ab4ae-7548-4ffd-a30d-262a4780c8e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.509558 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ab4ae-7548-4ffd-a30d-262a4780c8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.510251 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ab4ae-7548-4ffd-a30d-262a4780c8e4 admin admin] PATH_INFO: `/v3/projects/fdfe32930eb64a149f4dd6af361a9201/users/07d12ff9a6254fb9a32e1788f9920796/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:17.514065 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ab4ae-7548-4ffd-a30d-262a4780c8e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdfe32930eb64a149f4dd6af361a9201, user_id=07d12ff9a6254fb9a32e1788f9920796, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:17.517020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ab4ae-7548-4ffd-a30d-262a4780c8e4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:17.534135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 955/1919] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:17 2022] PUT /identity/v3/projects/fdfe32930eb64a149f4dd6af361a9201/users/07d12ff9a6254fb9a32e1788f9920796/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:17.545175 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbfac4-1868-4987-bc4a-7e88bfdcbee7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.545451 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbfac4-1868-4987-bc4a-7e88bfdcbee7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.545721 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbfac4-1868-4987-bc4a-7e88bfdcbee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:17.581932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-aafbfac4-1868-4987-bc4a-7e88bfdcbee7 None None] MFA Rules not processed for user `07d12ff9a6254fb9a32e1788f9920796`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:17.583760 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-aafbfac4-1868-4987-bc4a-7e88bfdcbee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:17.695018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 965/1920] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:17 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:17.703867 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ab4ae-7548-4ffd-a30d-262a4780c8e4 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:17.705649 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-917ab4ae-7548-4ffd-a30d-262a4780c8e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:17.781971 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ab4ae-7548-4ffd-a30d-262a4780c8e4 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:17.786305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1d373-40d5-489c-a0bd-c066dca86f97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:17.787137 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1d373-40d5-489c-a0bd-c066dca86f97 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.787410 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1d373-40d5-489c-a0bd-c066dca86f97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.787679 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1d373-40d5-489c-a0bd-c066dca86f97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:17.799778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f1d373-40d5-489c-a0bd-c066dca86f97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:17.802113 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f1d373-40d5-489c-a0bd-c066dca86f97 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:17.814192 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 956/1921] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:17 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:57:17.929804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbfac4-1868-4987-bc4a-7e88bfdcbee7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:17.940789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbfac4-1868-4987-bc4a-7e88bfdcbee7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:17.945509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f15f2f-9777-4a53-9e90-4917717ef710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:17.946150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f15f2f-9777-4a53-9e90-4917717ef710 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:17.946430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f15f2f-9777-4a53-9e90-4917717ef710 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:17.946693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f15f2f-9777-4a53-9e90-4917717ef710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:18.015076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f15f2f-9777-4a53-9e90-4917717ef710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:18.016573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f15f2f-9777-4a53-9e90-4917717ef710 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:18.027768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 966/1922] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:17 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:23.453027 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1d373-40d5-489c-a0bd-c066dca86f97 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:23.543476 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1d373-40d5-489c-a0bd-c066dca86f97 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:23.550136 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc0c77-8f56-4b0d-ad62-34dd3794c372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:23.553764 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc0c77-8f56-4b0d-ad62-34dd3794c372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:23.553764 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc0c77-8f56-4b0d-ad62-34dd3794c372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:23.553764 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc0c77-8f56-4b0d-ad62-34dd3794c372 admin admin] PATH_INFO: `/v3/users/4d78fca1ff3a4dbeb226e3b1c5869a8f` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:23.556061 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc0c77-8f56-4b0d-ad62-34dd3794c372 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d78fca1ff3a4dbeb226e3b1c5869a8f)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:23.558725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc0c77-8f56-4b0d-ad62-34dd3794c372 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:23.623414 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4ccc0c77-8f56-4b0d-ad62-34dd3794c372 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d78fca1ff3a4dbeb226e3b1c5869a8f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:23.634884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4ccc0c77-8f56-4b0d-ad62-34dd3794c372 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d78fca1ff3a4dbeb226e3b1c5869a8f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:23.652893 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4ccc0c77-8f56-4b0d-ad62-34dd3794c372 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d78fca1ff3a4dbeb226e3b1c5869a8f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:23.663465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4ccc0c77-8f56-4b0d-ad62-34dd3794c372 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d78fca1ff3a4dbeb226e3b1c5869a8f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:23.664498 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4ccc0c77-8f56-4b0d-ad62-34dd3794c372 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d78fca1ff3a4dbeb226e3b1c5869a8f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:23.671392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 957/1923] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:23 2022] DELETE /identity/v3/users/4d78fca1ff3a4dbeb226e3b1c5869a8f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:23.677238 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07e6d1-8adc-4d5d-8e57-88f439e594c4 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:23.677806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07e6d1-8adc-4d5d-8e57-88f439e594c4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:23.677806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07e6d1-8adc-4d5d-8e57-88f439e594c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:23.699657 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5e07e6d1-8adc-4d5d-8e57-88f439e594c4 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:23.700918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5e07e6d1-8adc-4d5d-8e57-88f439e594c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:23.763947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:23.789671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 967/1924] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:23.800918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc0c77-8f56-4b0d-ad62-34dd3794c372 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:23.803057 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4ccc0c77-8f56-4b0d-ad62-34dd3794c372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:23.830478 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:23.891915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07e6d1-8adc-4d5d-8e57-88f439e594c4 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:23.893528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5e07e6d1-8adc-4d5d-8e57-88f439e594c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:23.894107 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc0c77-8f56-4b0d-ad62-34dd3794c372 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:23.902607 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd9bb5-d8a5-4890-ae50-d90f44c00c9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:23.905129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd9bb5-d8a5-4890-ae50-d90f44c00c9e service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:23.906508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd9bb5-d8a5-4890-ae50-d90f44c00c9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:23.907253 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd9bb5-d8a5-4890-ae50-d90f44c00c9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:23.908779 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07e6d1-8adc-4d5d-8e57-88f439e594c4 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:23.914378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e33ac4-2108-4179-bc6c-cb4290f72b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:23.915232 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e33ac4-2108-4179-bc6c-cb4290f72b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:23.917195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e33ac4-2108-4179-bc6c-cb4290f72b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:23.917823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e33ac4-2108-4179-bc6c-cb4290f72b7d admin admin] PATH_INFO: `/v3/users/7b52e94c20d0419ea8e597bf85b91ed4` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:23.919724 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e33ac4-2108-4179-bc6c-cb4290f72b7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b52e94c20d0419ea8e597bf85b91ed4)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:23.922432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e33ac4-2108-4179-bc6c-cb4290f72b7d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:23.939958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd9bb5-d8a5-4890-ae50-d90f44c00c9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:23.939958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd9bb5-d8a5-4890-ae50-d90f44c00c9e service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:23.958736 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 958/1925] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:24.011461 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-92e33ac4-2108-4179-bc6c-cb4290f72b7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b52e94c20d0419ea8e597bf85b91ed4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.012647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-92e33ac4-2108-4179-bc6c-cb4290f72b7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b52e94c20d0419ea8e597bf85b91ed4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.024687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-92e33ac4-2108-4179-bc6c-cb4290f72b7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b52e94c20d0419ea8e597bf85b91ed4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.034961 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-92e33ac4-2108-4179-bc6c-cb4290f72b7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b52e94c20d0419ea8e597bf85b91ed4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.056243 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-92e33ac4-2108-4179-bc6c-cb4290f72b7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b52e94c20d0419ea8e597bf85b91ed4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.064893 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 968/1926] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:23 2022] DELETE /identity/v3/users/7b52e94c20d0419ea8e597bf85b91ed4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:24.071162 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491f1bf-f4be-4521-bc27-43fba5860a4c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:24.071899 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491f1bf-f4be-4521-bc27-43fba5860a4c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:24.072504 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491f1bf-f4be-4521-bc27-43fba5860a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:24.111780 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1491f1bf-f4be-4521-bc27-43fba5860a4c None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:24.115105 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1491f1bf-f4be-4521-bc27-43fba5860a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:24.235381 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 959/1927] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:24.249473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e33ac4-2108-4179-bc6c-cb4290f72b7d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:24.251695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-92e33ac4-2108-4179-bc6c-cb4290f72b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:24.287971 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491f1bf-f4be-4521-bc27-43fba5860a4c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:24.289788 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1491f1bf-f4be-4521-bc27-43fba5860a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:24.319213 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491f1bf-f4be-4521-bc27-43fba5860a4c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:24.324477 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73720944-c6de-49e6-bf03-e250df35d236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:24.325239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73720944-c6de-49e6-bf03-e250df35d236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:24.325594 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73720944-c6de-49e6-bf03-e250df35d236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:24.325946 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73720944-c6de-49e6-bf03-e250df35d236 admin admin] PATH_INFO: `/v3/projects/b639e83157e8404e9b1fd1ee9a8f591b` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:24.327374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73720944-c6de-49e6-bf03-e250df35d236 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b639e83157e8404e9b1fd1ee9a8f591b)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:24.329468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73720944-c6de-49e6-bf03-e250df35d236 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:24.387436 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e33ac4-2108-4179-bc6c-cb4290f72b7d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:24.393219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976b778-25bd-4a7a-9512-a99b7de35acb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:24.394042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976b778-25bd-4a7a-9512-a99b7de35acb service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:24.394429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976b778-25bd-4a7a-9512-a99b7de35acb service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:24.394907 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976b778-25bd-4a7a-9512-a99b7de35acb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:24.409836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8976b778-25bd-4a7a-9512-a99b7de35acb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:24.412413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8976b778-25bd-4a7a-9512-a99b7de35acb service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:24.425335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 969/1928] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:57:24.441534 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-73720944-c6de-49e6-bf03-e250df35d236 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b639e83157e8404e9b1fd1ee9a8f591b', 'request_id': 'req-73720944-c6de-49e6-bf03-e250df35d236'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.450453 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-73720944-c6de-49e6-bf03-e250df35d236 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b639e83157e8404e9b1fd1ee9a8f591b', 'request_id': 'req-73720944-c6de-49e6-bf03-e250df35d236'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.457920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-73720944-c6de-49e6-bf03-e250df35d236 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b639e83157e8404e9b1fd1ee9a8f591b', 'request_id': 'req-73720944-c6de-49e6-bf03-e250df35d236'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.472255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-73720944-c6de-49e6-bf03-e250df35d236 admin admin] The token cache is being invalidate because project b639e83157e8404e9b1fd1ee9a8f591b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:24.472888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-73720944-c6de-49e6-bf03-e250df35d236 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.474334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-73720944-c6de-49e6-bf03-e250df35d236 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.477114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 960/1929] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:24 2022] DELETE /identity/v3/projects/b639e83157e8404e9b1fd1ee9a8f591b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:24.481288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976b778-25bd-4a7a-9512-a99b7de35acb service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:24.497579 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976b778-25bd-4a7a-9512-a99b7de35acb service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:24.607311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bc77b-893b-4b66-826f-a78c546a74ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:24.607976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bc77b-893b-4b66-826f-a78c546a74ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:24.608292 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bc77b-893b-4b66-826f-a78c546a74ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:24.608763 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bc77b-893b-4b66-826f-a78c546a74ac admin admin] PATH_INFO: `/v3/users/07d12ff9a6254fb9a32e1788f9920796` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:24.610095 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bc77b-893b-4b66-826f-a78c546a74ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=07d12ff9a6254fb9a32e1788f9920796)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:24.611823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bc77b-893b-4b66-826f-a78c546a74ac admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:24.614513 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:24.704791 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-de7bc77b-893b-4b66-826f-a78c546a74ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07d12ff9a6254fb9a32e1788f9920796'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.706253 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-de7bc77b-893b-4b66-826f-a78c546a74ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07d12ff9a6254fb9a32e1788f9920796'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.725182 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-de7bc77b-893b-4b66-826f-a78c546a74ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07d12ff9a6254fb9a32e1788f9920796'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.735963 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-de7bc77b-893b-4b66-826f-a78c546a74ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07d12ff9a6254fb9a32e1788f9920796'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.761321 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-de7bc77b-893b-4b66-826f-a78c546a74ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07d12ff9a6254fb9a32e1788f9920796'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:24.775118 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 970/1930] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:24 2022] DELETE /identity/v3/users/07d12ff9a6254fb9a32e1788f9920796 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:24.804642 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73720944-c6de-49e6-bf03-e250df35d236 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:24.807168 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-73720944-c6de-49e6-bf03-e250df35d236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:24.826827 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73720944-c6de-49e6-bf03-e250df35d236 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:24.879802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:24.925475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408c9cd-246f-436a-8b94-7f5f7b96661f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:24.927276 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408c9cd-246f-436a-8b94-7f5f7b96661f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:24.928327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408c9cd-246f-436a-8b94-7f5f7b96661f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:24.929146 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408c9cd-246f-436a-8b94-7f5f7b96661f admin admin] PATH_INFO: `/v3/projects/6fc955bcd7af4bc09cfdc2f134ee827d` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:24.930996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408c9cd-246f-436a-8b94-7f5f7b96661f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fc955bcd7af4bc09cfdc2f134ee827d)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:24.933492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408c9cd-246f-436a-8b94-7f5f7b96661f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:25.040897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d408c9cd-246f-436a-8b94-7f5f7b96661f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fc955bcd7af4bc09cfdc2f134ee827d', 'request_id': 'req-d408c9cd-246f-436a-8b94-7f5f7b96661f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:25.054436 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d408c9cd-246f-436a-8b94-7f5f7b96661f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fc955bcd7af4bc09cfdc2f134ee827d', 'request_id': 'req-d408c9cd-246f-436a-8b94-7f5f7b96661f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:25.068557 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d408c9cd-246f-436a-8b94-7f5f7b96661f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fc955bcd7af4bc09cfdc2f134ee827d', 'request_id': 'req-d408c9cd-246f-436a-8b94-7f5f7b96661f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:25.091977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d408c9cd-246f-436a-8b94-7f5f7b96661f admin admin] The token cache is being invalidate because project 6fc955bcd7af4bc09cfdc2f134ee827d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:25.092310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d408c9cd-246f-436a-8b94-7f5f7b96661f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:25.093052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d408c9cd-246f-436a-8b94-7f5f7b96661f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:25.095313 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 961/1931] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:24 2022] DELETE /identity/v3/projects/6fc955bcd7af4bc09cfdc2f134ee827d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:25.101803 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a30b6-73d8-452e-9072-1f4e35829bae None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:25.102193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a30b6-73d8-452e-9072-1f4e35829bae None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:25.106913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a30b6-73d8-452e-9072-1f4e35829bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:25.172005 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408c9cd-246f-436a-8b94-7f5f7b96661f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:25.174275 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d408c9cd-246f-436a-8b94-7f5f7b96661f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:25.209119 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-948a30b6-73d8-452e-9072-1f4e35829bae None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:25.210269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-948a30b6-73d8-452e-9072-1f4e35829bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:25.262420 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408c9cd-246f-436a-8b94-7f5f7b96661f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:25.270834 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c6616-58cf-467a-8a7a-6ebe01bbcd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:25.270834 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c6616-58cf-467a-8a7a-6ebe01bbcd17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:25.270834 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c6616-58cf-467a-8a7a-6ebe01bbcd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:25.270834 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c6616-58cf-467a-8a7a-6ebe01bbcd17 admin admin] PATH_INFO: `/v3/projects/fdfe32930eb64a149f4dd6af361a9201` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:25.272980 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c6616-58cf-467a-8a7a-6ebe01bbcd17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdfe32930eb64a149f4dd6af361a9201)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:25.274317 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c6616-58cf-467a-8a7a-6ebe01bbcd17 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:25.300772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 971/1932] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:25.305165 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea587c-13d4-4cd7-83b9-f10588a20891 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:25.305165 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea587c-13d4-4cd7-83b9-f10588a20891 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:25.305165 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea587c-13d4-4cd7-83b9-f10588a20891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:25.330548 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-32ea587c-13d4-4cd7-83b9-f10588a20891 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:25.332218 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-32ea587c-13d4-4cd7-83b9-f10588a20891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:25.350134 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-639c6616-58cf-467a-8a7a-6ebe01bbcd17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdfe32930eb64a149f4dd6af361a9201', 'request_id': 'req-639c6616-58cf-467a-8a7a-6ebe01bbcd17'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:25.363670 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-639c6616-58cf-467a-8a7a-6ebe01bbcd17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdfe32930eb64a149f4dd6af361a9201', 'request_id': 'req-639c6616-58cf-467a-8a7a-6ebe01bbcd17'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:25.375395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-639c6616-58cf-467a-8a7a-6ebe01bbcd17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdfe32930eb64a149f4dd6af361a9201', 'request_id': 'req-639c6616-58cf-467a-8a7a-6ebe01bbcd17'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:25.388181 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:25.392950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-639c6616-58cf-467a-8a7a-6ebe01bbcd17 admin admin] The token cache is being invalidate because project fdfe32930eb64a149f4dd6af361a9201 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:25.394812 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-639c6616-58cf-467a-8a7a-6ebe01bbcd17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:25.397334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-639c6616-58cf-467a-8a7a-6ebe01bbcd17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:25.401755 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 962/1933] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:25 2022] DELETE /identity/v3/projects/fdfe32930eb64a149f4dd6af361a9201 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:25.406020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f6d91-fc6b-45d1-b005-07f79310dcd6 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:25.406020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f6d91-fc6b-45d1-b005-07f79310dcd6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:25.406650 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f6d91-fc6b-45d1-b005-07f79310dcd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:25.413326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 972/1934] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:25.418686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea587c-13d4-4cd7-83b9-f10588a20891 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:25.429747 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea587c-13d4-4cd7-83b9-f10588a20891 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:25.436567 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9054a-d60a-4c90-b3c6-57f98388d52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:25.437403 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9054a-d60a-4c90-b3c6-57f98388d52a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:25.438192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9054a-d60a-4c90-b3c6-57f98388d52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:25.438787 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9054a-d60a-4c90-b3c6-57f98388d52a admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:25.440049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9054a-d60a-4c90-b3c6-57f98388d52a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:25.440700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9054a-d60a-4c90-b3c6-57f98388d52a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:25.443449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9054a-d60a-4c90-b3c6-57f98388d52a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:25.451994 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-c89f6d91-fc6b-45d1-b005-07f79310dcd6 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:25.454699 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c89f6d91-fc6b-45d1-b005-07f79310dcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:25.479456 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 963/1935] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:25.496528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d9d89-8f83-4a24-8480-0baa3a69d5e3 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:25.496993 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d9d89-8f83-4a24-8480-0baa3a69d5e3 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:25.497552 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d9d89-8f83-4a24-8480-0baa3a69d5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:25.512885 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 973/1936] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:57:25.513824 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9054a-d60a-4c90-b3c6-57f98388d52a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:25.545173 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9054a-d60a-4c90-b3c6-57f98388d52a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:25.553055 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3b61d-6100-465f-9aad-795924a1e7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:25.553837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3b61d-6100-465f-9aad-795924a1e7f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:25.554122 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3b61d-6100-465f-9aad-795924a1e7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:25.554394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3b61d-6100-465f-9aad-795924a1e7f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:25.555042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3b61d-6100-465f-9aad-795924a1e7f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:25.557664 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3b61d-6100-465f-9aad-795924a1e7f5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:25.558948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-607d9d89-8f83-4a24-8480-0baa3a69d5e3 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:25.560345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-607d9d89-8f83-4a24-8480-0baa3a69d5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:25.562836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:25.585569 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 964/1937] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:25.588726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d9d89-8f83-4a24-8480-0baa3a69d5e3 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:25.602356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 974/1938] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:25 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:25.603597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3b61d-6100-465f-9aad-795924a1e7f5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:25.603946 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d9d89-8f83-4a24-8480-0baa3a69d5e3 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:25.683988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c8c9a-266c-4176-948c-bc006b2573a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:25.684674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c8c9a-266c-4176-948c-bc006b2573a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:25.685071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c8c9a-266c-4176-948c-bc006b2573a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:25.685383 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c8c9a-266c-4176-948c-bc006b2573a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:25.686045 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c8c9a-266c-4176-948c-bc006b2573a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:25.686377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c8c9a-266c-4176-948c-bc006b2573a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:25.688498 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c8c9a-266c-4176-948c-bc006b2573a9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:25.706079 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 965/1939] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:57:25.708614 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3b61d-6100-465f-9aad-795924a1e7f5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:25.710039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c8c9a-266c-4176-948c-bc006b2573a9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:25.718969 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24386c7a-2cf0-4ee3-80d8-e144d8ee09c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:25.720466 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24386c7a-2cf0-4ee3-80d8-e144d8ee09c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:25.720966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24386c7a-2cf0-4ee3-80d8-e144d8ee09c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:25.721597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24386c7a-2cf0-4ee3-80d8-e144d8ee09c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:25.722466 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24386c7a-2cf0-4ee3-80d8-e144d8ee09c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:25.725091 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c8c9a-266c-4176-948c-bc006b2573a9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:25.725777 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24386c7a-2cf0-4ee3-80d8-e144d8ee09c2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:25.732358 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa08c65-1f1e-49ae-a99e-07b44dde4964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:25.733003 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa08c65-1f1e-49ae-a99e-07b44dde4964 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:25.733285 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa08c65-1f1e-49ae-a99e-07b44dde4964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:25.733633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa08c65-1f1e-49ae-a99e-07b44dde4964 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:25.734502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa08c65-1f1e-49ae-a99e-07b44dde4964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:25.736546 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa08c65-1f1e-49ae-a99e-07b44dde4964 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:25.765596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 975/1940] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:25 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:25.770234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24386c7a-2cf0-4ee3-80d8-e144d8ee09c2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:25.791685 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 966/1941] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:25 2022] POST /identity/v3/projects => generated 377 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:25.791902 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24386c7a-2cf0-4ee3-80d8-e144d8ee09c2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:25.793219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa08c65-1f1e-49ae-a99e-07b44dde4964 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:25.870154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe538942-9a28-49d5-9410-dfabd33e5464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:25.871109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe538942-9a28-49d5-9410-dfabd33e5464 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:25.871522 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe538942-9a28-49d5-9410-dfabd33e5464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:25.871928 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe538942-9a28-49d5-9410-dfabd33e5464 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:25.872887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe538942-9a28-49d5-9410-dfabd33e5464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:25.874574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe538942-9a28-49d5-9410-dfabd33e5464 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:25.887596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 976/1942] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:25.891779 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe538942-9a28-49d5-9410-dfabd33e5464 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:25.907188 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe538942-9a28-49d5-9410-dfabd33e5464 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:25.919443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce09f27-fd9f-4f79-9829-8cd5a877e43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:25.920317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce09f27-fd9f-4f79-9829-8cd5a877e43e admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:25.920767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce09f27-fd9f-4f79-9829-8cd5a877e43e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:25.921088 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce09f27-fd9f-4f79-9829-8cd5a877e43e admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:25.922375 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce09f27-fd9f-4f79-9829-8cd5a877e43e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:25.924069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce09f27-fd9f-4f79-9829-8cd5a877e43e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:25.937730 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa08c65-1f1e-49ae-a99e-07b44dde4964 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:25.946760 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 977/1943] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:25.952522 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce09f27-fd9f-4f79-9829-8cd5a877e43e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:25.955371 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a82a4b-81e8-4f6e-994b-30d8f675bf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:25.958456 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a82a4b-81e8-4f6e-994b-30d8f675bf3f admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:25.960558 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a82a4b-81e8-4f6e-994b-30d8f675bf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:25.960921 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a82a4b-81e8-4f6e-994b-30d8f675bf3f admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:25.961964 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a82a4b-81e8-4f6e-994b-30d8f675bf3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:25.964528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a82a4b-81e8-4f6e-994b-30d8f675bf3f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:25.969287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce09f27-fd9f-4f79-9829-8cd5a877e43e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:25.976861 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ba1e6-ae00-4483-8adf-39fd925a05eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:25.977748 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ba1e6-ae00-4483-8adf-39fd925a05eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:25.978108 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ba1e6-ae00-4483-8adf-39fd925a05eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:25.978464 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ba1e6-ae00-4483-8adf-39fd925a05eb admin admin] PATH_INFO: `/v3/projects/61de8fb05c3a45de90ff3238b61452bb/users/3095e7f94a58497c812890fa86e08eec/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:25.998322 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ba1e6-ae00-4483-8adf-39fd925a05eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=61de8fb05c3a45de90ff3238b61452bb, user_id=3095e7f94a58497c812890fa86e08eec, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:26.000395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 967/1944] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:25 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:57:26.001398 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ba1e6-ae00-4483-8adf-39fd925a05eb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:26.009453 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a82a4b-81e8-4f6e-994b-30d8f675bf3f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:26.017236 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 978/1945] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:25 2022] PUT /identity/v3/projects/61de8fb05c3a45de90ff3238b61452bb/users/3095e7f94a58497c812890fa86e08eec/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:26.024919 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bea2e-0599-4c47-8632-c7f664a99f13 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:26.025437 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bea2e-0599-4c47-8632-c7f664a99f13 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:26.025866 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bea2e-0599-4c47-8632-c7f664a99f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:26.040602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a82a4b-81e8-4f6e-994b-30d8f675bf3f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:26.055077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-fa4bea2e-0599-4c47-8632-c7f664a99f13 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:26.056511 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fa4bea2e-0599-4c47-8632-c7f664a99f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:26.136151 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:26.154757 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:26.188368 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d8d9d-6189-4a0e-9440-5ca83ac8b646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:26.189022 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d8d9d-6189-4a0e-9440-5ca83ac8b646 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:26.189337 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d8d9d-6189-4a0e-9440-5ca83ac8b646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:26.189633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d8d9d-6189-4a0e-9440-5ca83ac8b646 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:26.190254 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d8d9d-6189-4a0e-9440-5ca83ac8b646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:26.191897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d8d9d-6189-4a0e-9440-5ca83ac8b646 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:26.212000 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 979/1946] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:26.213123 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 968/1947] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:26.214045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bea2e-0599-4c47-8632-c7f664a99f13 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:26.224755 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d8d9d-6189-4a0e-9440-5ca83ac8b646 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:26.227479 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-980d8d9d-6189-4a0e-9440-5ca83ac8b646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:26.235957 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bea2e-0599-4c47-8632-c7f664a99f13 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:26.247457 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30a03d-2fb8-4154-93e3-61ae49065be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:26.248276 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30a03d-2fb8-4154-93e3-61ae49065be3 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:26.248655 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30a03d-2fb8-4154-93e3-61ae49065be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:26.249043 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30a03d-2fb8-4154-93e3-61ae49065be3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:26.249965 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30a03d-2fb8-4154-93e3-61ae49065be3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:26.251814 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30a03d-2fb8-4154-93e3-61ae49065be3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:26.264751 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 980/1948] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:26.269681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30a03d-2fb8-4154-93e3-61ae49065be3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:26.284573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30a03d-2fb8-4154-93e3-61ae49065be3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:26.291256 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87f82c-4921-486b-8f78-afc1443730de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:26.291953 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87f82c-4921-486b-8f78-afc1443730de admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:26.292283 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87f82c-4921-486b-8f78-afc1443730de admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:26.292574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87f82c-4921-486b-8f78-afc1443730de admin admin] PATH_INFO: `/v3/projects/5b2b49a591da4636a9785ff8e17a0c2c/users/c42d53043b6d4080a9e4dad364f0ce12/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:26.312410 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87f82c-4921-486b-8f78-afc1443730de admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b2b49a591da4636a9785ff8e17a0c2c, user_id=c42d53043b6d4080a9e4dad364f0ce12, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:26.315815 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87f82c-4921-486b-8f78-afc1443730de admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:26.322629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d8d9d-6189-4a0e-9440-5ca83ac8b646 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:26.327231 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe73c6-604b-4967-ac4f-23b51320023a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:26.328025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe73c6-604b-4967-ac4f-23b51320023a service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:26.328581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe73c6-604b-4967-ac4f-23b51320023a service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:26.329065 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe73c6-604b-4967-ac4f-23b51320023a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:26.341046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe73c6-604b-4967-ac4f-23b51320023a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:26.344087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe73c6-604b-4967-ac4f-23b51320023a service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:26.351013 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 981/1949] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:26 2022] PUT /identity/v3/projects/5b2b49a591da4636a9785ff8e17a0c2c/users/c42d53043b6d4080a9e4dad364f0ce12/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:26.352370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b06875-8aa5-4dfc-b0ee-24781084643a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:26.352664 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b06875-8aa5-4dfc-b0ee-24781084643a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:26.352982 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b06875-8aa5-4dfc-b0ee-24781084643a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:26.361059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 969/1950] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:26.403341 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-57b06875-8aa5-4dfc-b0ee-24781084643a None None] MFA Rules not processed for user `c42d53043b6d4080a9e4dad364f0ce12`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:26.404658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-57b06875-8aa5-4dfc-b0ee-24781084643a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:26.516859 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 982/1951] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:26.523085 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe73c6-604b-4967-ac4f-23b51320023a service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:26.543531 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe73c6-604b-4967-ac4f-23b51320023a service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:26.666639 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a0f0b-189a-4c88-b34d-6891836f6412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:26.667394 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a0f0b-189a-4c88-b34d-6891836f6412 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:26.668262 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a0f0b-189a-4c88-b34d-6891836f6412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:26.668602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a0f0b-189a-4c88-b34d-6891836f6412 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:26.670351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a0f0b-189a-4c88-b34d-6891836f6412 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:26.672539 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a0f0b-189a-4c88-b34d-6891836f6412 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:26.709496 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 970/1952] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:26 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:26.713801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b06875-8aa5-4dfc-b0ee-24781084643a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:26.730343 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b06875-8aa5-4dfc-b0ee-24781084643a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:26.821062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da20961-b1c1-423f-9cc8-90723ddd733c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:26.821730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da20961-b1c1-423f-9cc8-90723ddd733c admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:26.822012 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da20961-b1c1-423f-9cc8-90723ddd733c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:26.822338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da20961-b1c1-423f-9cc8-90723ddd733c admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:26.822951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da20961-b1c1-423f-9cc8-90723ddd733c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:26.825153 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da20961-b1c1-423f-9cc8-90723ddd733c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:26.850945 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 983/1953] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:26 2022] POST /identity/v3/users => generated 298 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:57:26.854647 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a0f0b-189a-4c88-b34d-6891836f6412 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:26.869399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a0f0b-189a-4c88-b34d-6891836f6412 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:26.877124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7bfe6b-b764-49f8-8a29-b4859216180d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:26.877124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bfe6b-b764-49f8-8a29-b4859216180d admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:26.877124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bfe6b-b764-49f8-8a29-b4859216180d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:26.877537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bfe6b-b764-49f8-8a29-b4859216180d admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:26.877943 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7bfe6b-b764-49f8-8a29-b4859216180d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:26.879677 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7bfe6b-b764-49f8-8a29-b4859216180d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:26.894795 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 971/1954] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:26.901065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da20961-b1c1-423f-9cc8-90723ddd733c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:26.911809 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da20961-b1c1-423f-9cc8-90723ddd733c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:26.917153 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f10b9b-fcd8-499f-b35d-89b1f9b64600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:26.917894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f10b9b-fcd8-499f-b35d-89b1f9b64600 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:26.918182 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f10b9b-fcd8-499f-b35d-89b1f9b64600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:26.918458 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f10b9b-fcd8-499f-b35d-89b1f9b64600 admin admin] PATH_INFO: `/v3/projects/a53311f9750941d793659081c3994066/users/b36ff49febc44887b9a496e4ddd48204/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:26.937801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f10b9b-fcd8-499f-b35d-89b1f9b64600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a53311f9750941d793659081c3994066, user_id=b36ff49febc44887b9a496e4ddd48204, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:26.940601 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f10b9b-fcd8-499f-b35d-89b1f9b64600 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:26.944894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:26.964880 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 984/1955] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:26 2022] PUT /identity/v3/projects/a53311f9750941d793659081c3994066/users/b36ff49febc44887b9a496e4ddd48204/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:26.968589 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7bfe6b-b764-49f8-8a29-b4859216180d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:26.986356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7bfe6b-b764-49f8-8a29-b4859216180d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:27.056979 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948b4c3-a9f7-49b0-9122-659ad74ed817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:27.058476 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948b4c3-a9f7-49b0-9122-659ad74ed817 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:27.058767 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948b4c3-a9f7-49b0-9122-659ad74ed817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:27.059102 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948b4c3-a9f7-49b0-9122-659ad74ed817 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:27.059696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948b4c3-a9f7-49b0-9122-659ad74ed817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:27.063545 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948b4c3-a9f7-49b0-9122-659ad74ed817 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:27.065524 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:27.081106 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 972/1956] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:27.083736 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f10b9b-fcd8-499f-b35d-89b1f9b64600 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:27.094915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f10b9b-fcd8-499f-b35d-89b1f9b64600 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:27.101250 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448cfda7-71c3-48e5-a80e-17a9c7617212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:27.101821 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cfda7-71c3-48e5-a80e-17a9c7617212 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:27.102102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cfda7-71c3-48e5-a80e-17a9c7617212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:27.102384 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cfda7-71c3-48e5-a80e-17a9c7617212 admin admin] PATH_INFO: `/v3/projects/a53311f9750941d793659081c3994066/users/b36ff49febc44887b9a496e4ddd48204/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:27.105122 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448cfda7-71c3-48e5-a80e-17a9c7617212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a53311f9750941d793659081c3994066, user_id=b36ff49febc44887b9a496e4ddd48204, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:27.108887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448cfda7-71c3-48e5-a80e-17a9c7617212 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:27.123939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 985/1957] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:27 2022] PUT /identity/v3/projects/a53311f9750941d793659081c3994066/users/b36ff49febc44887b9a496e4ddd48204/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:27.134308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a62ea2-4666-4b1a-b91e-c8d17589a028 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:27.134606 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a62ea2-4666-4b1a-b91e-c8d17589a028 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:27.134881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a62ea2-4666-4b1a-b91e-c8d17589a028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:27.168776 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-96a62ea2-4666-4b1a-b91e-c8d17589a028 None None] MFA Rules not processed for user `b36ff49febc44887b9a496e4ddd48204`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:27.169928 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-96a62ea2-4666-4b1a-b91e-c8d17589a028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:27.202036 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db376f65-c752-43f8-a73b-2615a3552bcd None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:27.202320 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db376f65-c752-43f8-a73b-2615a3552bcd None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:27.202599 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db376f65-c752-43f8-a73b-2615a3552bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:27.236857 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-db376f65-c752-43f8-a73b-2615a3552bcd None None] MFA Rules not processed for user `3095e7f94a58497c812890fa86e08eec`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:27.238357 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-db376f65-c752-43f8-a73b-2615a3552bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:27.264132 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 973/1958] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:27.271206 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a62ea2-4666-4b1a-b91e-c8d17589a028 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:27.273200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-96a62ea2-4666-4b1a-b91e-c8d17589a028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:27.330298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 986/1959] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:27.341861 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db376f65-c752-43f8-a73b-2615a3552bcd None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:27.344113 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-db376f65-c752-43f8-a73b-2615a3552bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:27.356723 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a62ea2-4666-4b1a-b91e-c8d17589a028 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:27.363596 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7acd659-143b-4d8f-a60b-aa683834b12f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:27.364399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acd659-143b-4d8f-a60b-aa683834b12f service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:27.364745 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acd659-143b-4d8f-a60b-aa683834b12f service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:27.365078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acd659-143b-4d8f-a60b-aa683834b12f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:27.369566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db376f65-c752-43f8-a73b-2615a3552bcd None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:27.383918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a23df-adb3-4bca-bc97-735efb3e6d96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:27.384593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a23df-adb3-4bca-bc97-735efb3e6d96 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:27.384871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a23df-adb3-4bca-bc97-735efb3e6d96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:27.385140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a23df-adb3-4bca-bc97-735efb3e6d96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:27.387825 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7acd659-143b-4d8f-a60b-aa683834b12f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:27.389404 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7acd659-143b-4d8f-a60b-aa683834b12f service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:27.402868 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 974/1960] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:57:27.418987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a23df-adb3-4bca-bc97-735efb3e6d96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:27.420506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a23df-adb3-4bca-bc97-735efb3e6d96 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:27.450413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 987/1961] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:27.585744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7acd659-143b-4d8f-a60b-aa683834b12f service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:27.597467 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7acd659-143b-4d8f-a60b-aa683834b12f service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:27.603525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912b8ec-4418-4320-8d60-d81458dd2410 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:27.604688 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912b8ec-4418-4320-8d60-d81458dd2410 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:27.604965 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912b8ec-4418-4320-8d60-d81458dd2410 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:27.605235 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912b8ec-4418-4320-8d60-d81458dd2410 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:27.687635 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912b8ec-4418-4320-8d60-d81458dd2410 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:27.689138 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912b8ec-4418-4320-8d60-d81458dd2410 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:27.705537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 975/1962] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:57:29.033873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dbb7a-2223-4e58-b1e1-5cfdab0c6189 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:29.034522 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dbb7a-2223-4e58-b1e1-5cfdab0c6189 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:29.034522 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dbb7a-2223-4e58-b1e1-5cfdab0c6189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:29.053825 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d28dbb7a-2223-4e58-b1e1-5cfdab0c6189 None None] MFA Rules not processed for user `3095e7f94a58497c812890fa86e08eec`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:29.055023 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d28dbb7a-2223-4e58-b1e1-5cfdab0c6189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:29.062037 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 988/1963] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:37.698745 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=90 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:38.106059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912b8ec-4418-4320-8d60-d81458dd2410 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:38.201036 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912b8ec-4418-4320-8d60-d81458dd2410 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:38.205610 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d572b02-f68d-4894-8707-275486455ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:38.206255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d572b02-f68d-4894-8707-275486455ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:38.206538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d572b02-f68d-4894-8707-275486455ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:38.206812 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d572b02-f68d-4894-8707-275486455ab8 admin admin] PATH_INFO: `/v3/users/c42d53043b6d4080a9e4dad364f0ce12` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:38.207945 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d572b02-f68d-4894-8707-275486455ab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c42d53043b6d4080a9e4dad364f0ce12)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:38.210409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d572b02-f68d-4894-8707-275486455ab8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:38.273699 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8d572b02-f68d-4894-8707-275486455ab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c42d53043b6d4080a9e4dad364f0ce12'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:38.282534 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8d572b02-f68d-4894-8707-275486455ab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c42d53043b6d4080a9e4dad364f0ce12'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:38.299775 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8d572b02-f68d-4894-8707-275486455ab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c42d53043b6d4080a9e4dad364f0ce12'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:38.312990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8d572b02-f68d-4894-8707-275486455ab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c42d53043b6d4080a9e4dad364f0ce12'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:38.314151 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8d572b02-f68d-4894-8707-275486455ab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c42d53043b6d4080a9e4dad364f0ce12'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:38.324232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 976/1964] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:38 2022] DELETE /identity/v3/users/c42d53043b6d4080a9e4dad364f0ce12 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:38.328922 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbcc2d-ad06-494f-8244-be83f542d9ea None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:38.329744 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbcc2d-ad06-494f-8244-be83f542d9ea None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:38.329744 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbcc2d-ad06-494f-8244-be83f542d9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:38.349017 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-19cbcc2d-ad06-494f-8244-be83f542d9ea None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:38.350257 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-19cbcc2d-ad06-494f-8244-be83f542d9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:38.422355 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 989/1965] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:38.433802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d572b02-f68d-4894-8707-275486455ab8 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:38.435432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8d572b02-f68d-4894-8707-275486455ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:38.548696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d572b02-f68d-4894-8707-275486455ab8 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:38.555551 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2444d9-4a83-4640-9e6b-eedf452c45c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:38.556592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2444d9-4a83-4640-9e6b-eedf452c45c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:38.556929 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2444d9-4a83-4640-9e6b-eedf452c45c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:38.557370 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2444d9-4a83-4640-9e6b-eedf452c45c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:38.574029 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2444d9-4a83-4640-9e6b-eedf452c45c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:38.576459 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2444d9-4a83-4640-9e6b-eedf452c45c6 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:38.589107 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 977/1966] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:38.998836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cbcc2d-ad06-494f-8244-be83f542d9ea None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:39.000480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-19cbcc2d-ad06-494f-8244-be83f542d9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:39.015468 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cbcc2d-ad06-494f-8244-be83f542d9ea None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:39.021979 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a1f31-a6ab-4a6f-9b6f-db4e36a5f1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:39.022637 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a1f31-a6ab-4a6f-9b6f-db4e36a5f1d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:39.022913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a1f31-a6ab-4a6f-9b6f-db4e36a5f1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:39.023185 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a1f31-a6ab-4a6f-9b6f-db4e36a5f1d0 admin admin] PATH_INFO: `/v3/projects/5b2b49a591da4636a9785ff8e17a0c2c` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:39.024370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a1f31-a6ab-4a6f-9b6f-db4e36a5f1d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b2b49a591da4636a9785ff8e17a0c2c)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:39.026366 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a1f31-a6ab-4a6f-9b6f-db4e36a5f1d0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:39.077834 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:39.079193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-302a1f31-a6ab-4a6f-9b6f-db4e36a5f1d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b2b49a591da4636a9785ff8e17a0c2c', 'request_id': 'req-302a1f31-a6ab-4a6f-9b6f-db4e36a5f1d0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:39.086015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-302a1f31-a6ab-4a6f-9b6f-db4e36a5f1d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b2b49a591da4636a9785ff8e17a0c2c', 'request_id': 'req-302a1f31-a6ab-4a6f-9b6f-db4e36a5f1d0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:39.099368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-302a1f31-a6ab-4a6f-9b6f-db4e36a5f1d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b2b49a591da4636a9785ff8e17a0c2c', 'request_id': 'req-302a1f31-a6ab-4a6f-9b6f-db4e36a5f1d0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:39.112193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-302a1f31-a6ab-4a6f-9b6f-db4e36a5f1d0 admin admin] The token cache is being invalidate because project 5b2b49a591da4636a9785ff8e17a0c2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:39.112424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-302a1f31-a6ab-4a6f-9b6f-db4e36a5f1d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:39.113082 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-302a1f31-a6ab-4a6f-9b6f-db4e36a5f1d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:39.116180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 990/1967] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:38 2022] DELETE /identity/v3/projects/5b2b49a591da4636a9785ff8e17a0c2c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:39.119216 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2444d9-4a83-4640-9e6b-eedf452c45c6 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:39.129644 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2444d9-4a83-4640-9e6b-eedf452c45c6 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:39.154380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:39.205776 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c1264-8f49-4201-b914-50709053743c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:39.206431 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c1264-8f49-4201-b914-50709053743c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:39.206720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c1264-8f49-4201-b914-50709053743c admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:39.206981 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c1264-8f49-4201-b914-50709053743c admin admin] PATH_INFO: `/v3/users/b36ff49febc44887b9a496e4ddd48204` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:39.208189 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c1264-8f49-4201-b914-50709053743c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b36ff49febc44887b9a496e4ddd48204)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:39.210358 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c1264-8f49-4201-b914-50709053743c admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:39.265826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bd6c1264-8f49-4201-b914-50709053743c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b36ff49febc44887b9a496e4ddd48204'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:39.274239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bd6c1264-8f49-4201-b914-50709053743c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b36ff49febc44887b9a496e4ddd48204'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:39.288633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bd6c1264-8f49-4201-b914-50709053743c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b36ff49febc44887b9a496e4ddd48204'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:39.296422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bd6c1264-8f49-4201-b914-50709053743c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b36ff49febc44887b9a496e4ddd48204'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:39.297152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bd6c1264-8f49-4201-b914-50709053743c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b36ff49febc44887b9a496e4ddd48204'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:39.303887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 978/1968] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:39 2022] DELETE /identity/v3/users/b36ff49febc44887b9a496e4ddd48204 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:39.610924 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a1f31-a6ab-4a6f-9b6f-db4e36a5f1d0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:39.612559 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-302a1f31-a6ab-4a6f-9b6f-db4e36a5f1d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:39.633449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a1f31-a6ab-4a6f-9b6f-db4e36a5f1d0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:39.699857 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fbff9-4b88-41fe-923f-fdfb0a7a875b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:39.701579 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fbff9-4b88-41fe-923f-fdfb0a7a875b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:39.702136 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fbff9-4b88-41fe-923f-fdfb0a7a875b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:39.702627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fbff9-4b88-41fe-923f-fdfb0a7a875b admin admin] PATH_INFO: `/v3/projects/a53311f9750941d793659081c3994066` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:39.704858 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2fbff9-4b88-41fe-923f-fdfb0a7a875b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a53311f9750941d793659081c3994066)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:39.708764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2fbff9-4b88-41fe-923f-fdfb0a7a875b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:39.783039 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bd2fbff9-4b88-41fe-923f-fdfb0a7a875b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a53311f9750941d793659081c3994066', 'request_id': 'req-bd2fbff9-4b88-41fe-923f-fdfb0a7a875b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:39.790277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bd2fbff9-4b88-41fe-923f-fdfb0a7a875b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a53311f9750941d793659081c3994066', 'request_id': 'req-bd2fbff9-4b88-41fe-923f-fdfb0a7a875b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:39.799536 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bd2fbff9-4b88-41fe-923f-fdfb0a7a875b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a53311f9750941d793659081c3994066', 'request_id': 'req-bd2fbff9-4b88-41fe-923f-fdfb0a7a875b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:39.814170 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bd2fbff9-4b88-41fe-923f-fdfb0a7a875b admin admin] The token cache is being invalidate because project a53311f9750941d793659081c3994066 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:39.814463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bd2fbff9-4b88-41fe-923f-fdfb0a7a875b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:39.815146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bd2fbff9-4b88-41fe-923f-fdfb0a7a875b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:39.816712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 991/1969] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:39 2022] DELETE /identity/v3/projects/a53311f9750941d793659081c3994066 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:39.822693 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad8bb4-01eb-4bfa-9fb8-804843f7afef None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:39.823170 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad8bb4-01eb-4bfa-9fb8-804843f7afef None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:39.823478 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad8bb4-01eb-4bfa-9fb8-804843f7afef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:39.848506 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d5ad8bb4-01eb-4bfa-9fb8-804843f7afef None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:39.850539 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d5ad8bb4-01eb-4bfa-9fb8-804843f7afef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:39.942769 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 979/1970] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:39.947021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2abc1b-2cda-4d32-9620-5ad0597f2482 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:39.947389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2abc1b-2cda-4d32-9620-5ad0597f2482 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:39.947768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2abc1b-2cda-4d32-9620-5ad0597f2482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:39.975605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ef2abc1b-2cda-4d32-9620-5ad0597f2482 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:39.978119 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ef2abc1b-2cda-4d32-9620-5ad0597f2482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:40.001617 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 992/1971] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:40.002973 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad8bb4-01eb-4bfa-9fb8-804843f7afef None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:40.018397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad8bb4-01eb-4bfa-9fb8-804843f7afef None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:40.029112 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec9727-aeb7-482c-a556-195a77a23bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:40.029985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec9727-aeb7-482c-a556-195a77a23bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.030393 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec9727-aeb7-482c-a556-195a77a23bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.030900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec9727-aeb7-482c-a556-195a77a23bd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.031967 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec9727-aeb7-482c-a556-195a77a23bd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:40.032408 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec9727-aeb7-482c-a556-195a77a23bd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:40.035245 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec9727-aeb7-482c-a556-195a77a23bd6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:40.059236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 980/1972] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:57:40.067406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2abc1b-2cda-4d32-9620-5ad0597f2482 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:40.083107 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2abc1b-2cda-4d32-9620-5ad0597f2482 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:40.091971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a401a5-aa9b-4110-93cc-72b8c5c266c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:40.091971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a401a5-aa9b-4110-93cc-72b8c5c266c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.091971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a401a5-aa9b-4110-93cc-72b8c5c266c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.093007 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a401a5-aa9b-4110-93cc-72b8c5c266c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.093007 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a401a5-aa9b-4110-93cc-72b8c5c266c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:40.095225 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a401a5-aa9b-4110-93cc-72b8c5c266c6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:40.137450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 993/1973] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:40 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:40.142082 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec9727-aeb7-482c-a556-195a77a23bd6 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:40.156992 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec9727-aeb7-482c-a556-195a77a23bd6 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:40.199554 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:40.257865 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53e6a9-93c1-489c-8353-2a3d890c04ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:40.258612 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53e6a9-93c1-489c-8353-2a3d890c04ac admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.258930 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53e6a9-93c1-489c-8353-2a3d890c04ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.259204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53e6a9-93c1-489c-8353-2a3d890c04ac admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.259839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53e6a9-93c1-489c-8353-2a3d890c04ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:40.261898 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53e6a9-93c1-489c-8353-2a3d890c04ac admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:40.294821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 981/1974] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:40 2022] POST /identity/v3/users => generated 289 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:40.298228 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a401a5-aa9b-4110-93cc-72b8c5c266c6 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:40.318960 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a401a5-aa9b-4110-93cc-72b8c5c266c6 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:40.334795 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15dcf6-a7ca-4aa6-879e-f752e1b4ea89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:40.335589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15dcf6-a7ca-4aa6-879e-f752e1b4ea89 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.335913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15dcf6-a7ca-4aa6-879e-f752e1b4ea89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.336204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15dcf6-a7ca-4aa6-879e-f752e1b4ea89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.336864 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15dcf6-a7ca-4aa6-879e-f752e1b4ea89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:40.342703 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15dcf6-a7ca-4aa6-879e-f752e1b4ea89 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:40.361400 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 994/1975] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:40.367504 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53e6a9-93c1-489c-8353-2a3d890c04ac admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:40.378949 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53e6a9-93c1-489c-8353-2a3d890c04ac admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:40.388269 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b077a-939f-4eee-b14d-bbe08b7d0e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:40.390841 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b077a-939f-4eee-b14d-bbe08b7d0e64 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.391446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b077a-939f-4eee-b14d-bbe08b7d0e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.391836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b077a-939f-4eee-b14d-bbe08b7d0e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.393660 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b077a-939f-4eee-b14d-bbe08b7d0e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:40.395654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b077a-939f-4eee-b14d-bbe08b7d0e64 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:40.414624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 982/1976] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:40.419538 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15dcf6-a7ca-4aa6-879e-f752e1b4ea89 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:40.439219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15dcf6-a7ca-4aa6-879e-f752e1b4ea89 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:40.454119 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af2cf5-fb8c-4502-9fdf-4b780a0e97b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:40.455802 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af2cf5-fb8c-4502-9fdf-4b780a0e97b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.456214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af2cf5-fb8c-4502-9fdf-4b780a0e97b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.456557 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af2cf5-fb8c-4502-9fdf-4b780a0e97b2 admin admin] PATH_INFO: `/v3/projects/d473cb5f9a61464c8bc8034d24bdf1b5/users/52a638e8e76a4bcf8b5c5ac8cc1f5efe/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.479365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af2cf5-fb8c-4502-9fdf-4b780a0e97b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d473cb5f9a61464c8bc8034d24bdf1b5, user_id=52a638e8e76a4bcf8b5c5ac8cc1f5efe, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:40.482163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af2cf5-fb8c-4502-9fdf-4b780a0e97b2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:40.504779 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 995/1977] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:40 2022] PUT /identity/v3/projects/d473cb5f9a61464c8bc8034d24bdf1b5/users/52a638e8e76a4bcf8b5c5ac8cc1f5efe/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:40.515265 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3740e-2162-451f-a78e-3f8bf636a85a None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.515595 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3740e-2162-451f-a78e-3f8bf636a85a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.515819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3740e-2162-451f-a78e-3f8bf636a85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.569789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-0da3740e-2162-451f-a78e-3f8bf636a85a None None] MFA Rules not processed for user `52a638e8e76a4bcf8b5c5ac8cc1f5efe`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:40.571211 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0da3740e-2162-451f-a78e-3f8bf636a85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:40.650556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 983/1978] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:40.655995 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af2cf5-fb8c-4502-9fdf-4b780a0e97b2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:40.662058 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:40.735609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af2cf5-fb8c-4502-9fdf-4b780a0e97b2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:40.740382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d7ff5-78cc-48cb-88ad-430dafe20b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:40.741857 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d7ff5-78cc-48cb-88ad-430dafe20b4d admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.741857 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d7ff5-78cc-48cb-88ad-430dafe20b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.742049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d7ff5-78cc-48cb-88ad-430dafe20b4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.742670 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d7ff5-78cc-48cb-88ad-430dafe20b4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:40.744870 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d7ff5-78cc-48cb-88ad-430dafe20b4d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:40.773587 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 996/1979] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:40 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:40.777945 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3740e-2162-451f-a78e-3f8bf636a85a None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:40.789602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3740e-2162-451f-a78e-3f8bf636a85a None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:40.851057 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc912c-519e-4b53-9952-bb38b005dc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:40.851683 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc912c-519e-4b53-9952-bb38b005dc4a admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.851968 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc912c-519e-4b53-9952-bb38b005dc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.852284 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc912c-519e-4b53-9952-bb38b005dc4a admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.852928 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc912c-519e-4b53-9952-bb38b005dc4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:40.854921 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc912c-519e-4b53-9952-bb38b005dc4a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:40.877897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 984/1980] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:40 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:40.882815 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d7ff5-78cc-48cb-88ad-430dafe20b4d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:40.893555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d7ff5-78cc-48cb-88ad-430dafe20b4d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:40.899234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb5366-53b4-4c1b-8c63-6bbaab728aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:40.899924 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb5366-53b4-4c1b-8c63-6bbaab728aef admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.900249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb5366-53b4-4c1b-8c63-6bbaab728aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.900524 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb5366-53b4-4c1b-8c63-6bbaab728aef admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.901174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb5366-53b4-4c1b-8c63-6bbaab728aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:40.902791 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb5366-53b4-4c1b-8c63-6bbaab728aef admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:40.914482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 997/1981] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:40.918580 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc912c-519e-4b53-9952-bb38b005dc4a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:40.933151 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc912c-519e-4b53-9952-bb38b005dc4a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:40.940259 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593354b6-0290-4f15-bd33-3b08db7a6e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:40.940896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593354b6-0290-4f15-bd33-3b08db7a6e8f admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:40.941168 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593354b6-0290-4f15-bd33-3b08db7a6e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:40.941443 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593354b6-0290-4f15-bd33-3b08db7a6e8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:40.942125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593354b6-0290-4f15-bd33-3b08db7a6e8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:40.949027 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593354b6-0290-4f15-bd33-3b08db7a6e8f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:40.973019 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 985/1982] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:40.978169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb5366-53b4-4c1b-8c63-6bbaab728aef admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:40.993083 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb5366-53b4-4c1b-8c63-6bbaab728aef admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:41.000977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58865d-344d-4c47-8ee3-460263a5d750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:41.001762 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58865d-344d-4c47-8ee3-460263a5d750 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:41.002204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58865d-344d-4c47-8ee3-460263a5d750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:41.002521 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58865d-344d-4c47-8ee3-460263a5d750 admin admin] PATH_INFO: `/v3/projects/6368a76730304cd5a71b863a7d5f86e0/users/56b80e932a264d028a1018c7cfdeed46/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:41.021849 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c58865d-344d-4c47-8ee3-460263a5d750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6368a76730304cd5a71b863a7d5f86e0, user_id=56b80e932a264d028a1018c7cfdeed46, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:41.024732 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c58865d-344d-4c47-8ee3-460263a5d750 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:41.047138 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 998/1983] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:40 2022] PUT /identity/v3/projects/6368a76730304cd5a71b863a7d5f86e0/users/56b80e932a264d028a1018c7cfdeed46/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:41.059427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc5f06-7583-47c3-a8bb-b356abce7c73 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:41.059769 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc5f06-7583-47c3-a8bb-b356abce7c73 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:41.060063 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc5f06-7583-47c3-a8bb-b356abce7c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:41.108645 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-cdcc5f06-7583-47c3-a8bb-b356abce7c73 None None] MFA Rules not processed for user `56b80e932a264d028a1018c7cfdeed46`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:41.110455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-cdcc5f06-7583-47c3-a8bb-b356abce7c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:41.117945 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:41.233292 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 986/1984] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:41.236523 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58865d-344d-4c47-8ee3-460263a5d750 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:41.453539 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58865d-344d-4c47-8ee3-460263a5d750 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:41.458874 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6d7f3-b005-4a59-9e94-4c320d58a20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:41.459558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6d7f3-b005-4a59-9e94-4c320d58a20c admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:41.459847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6d7f3-b005-4a59-9e94-4c320d58a20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:41.460192 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6d7f3-b005-4a59-9e94-4c320d58a20c admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:41.464261 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6d7f3-b005-4a59-9e94-4c320d58a20c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:41.466247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6d7f3-b005-4a59-9e94-4c320d58a20c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:41.514153 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 999/1985] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:41 2022] POST /identity/v3/projects => generated 373 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:41.518904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc5f06-7583-47c3-a8bb-b356abce7c73 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:41.539671 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc5f06-7583-47c3-a8bb-b356abce7c73 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:41.634723 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e7437-ff29-4b47-adc5-1300066d6d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:41.635384 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e7437-ff29-4b47-adc5-1300066d6d68 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:41.635668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e7437-ff29-4b47-adc5-1300066d6d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:41.635947 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e7437-ff29-4b47-adc5-1300066d6d68 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:41.636857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e7437-ff29-4b47-adc5-1300066d6d68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:41.639128 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e7437-ff29-4b47-adc5-1300066d6d68 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:41.681337 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 987/1986] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:41 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:57:41.682893 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6d7f3-b005-4a59-9e94-4c320d58a20c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:41.703163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6d7f3-b005-4a59-9e94-4c320d58a20c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:41.732482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5ab45-7aa8-4856-a6fa-c6b3e55228fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:41.734040 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5ab45-7aa8-4856-a6fa-c6b3e55228fc admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:41.734787 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5ab45-7aa8-4856-a6fa-c6b3e55228fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:41.735491 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5ab45-7aa8-4856-a6fa-c6b3e55228fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:41.738889 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5ab45-7aa8-4856-a6fa-c6b3e55228fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:41.749291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5ab45-7aa8-4856-a6fa-c6b3e55228fc admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:41.798456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1000/1987] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:41.805114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e7437-ff29-4b47-adc5-1300066d6d68 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:41.824385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e7437-ff29-4b47-adc5-1300066d6d68 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:41.840790 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b89c56-a2d8-4118-841a-22504c7de94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:41.842149 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b89c56-a2d8-4118-841a-22504c7de94b admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:41.842676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b89c56-a2d8-4118-841a-22504c7de94b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:41.843158 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b89c56-a2d8-4118-841a-22504c7de94b admin admin] PATH_INFO: `/v3/projects/3bc6c32abb8142c98896ccbd7f634718/users/92edc0064fb6489d847b4d1206eccdfa/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:41.884970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b89c56-a2d8-4118-841a-22504c7de94b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc6c32abb8142c98896ccbd7f634718, user_id=92edc0064fb6489d847b4d1206eccdfa, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:41.888301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b89c56-a2d8-4118-841a-22504c7de94b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:41.931229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 988/1988] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:41 2022] PUT /identity/v3/projects/3bc6c32abb8142c98896ccbd7f634718/users/92edc0064fb6489d847b4d1206eccdfa/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:41.935118 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5ab45-7aa8-4856-a6fa-c6b3e55228fc admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:41.949421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5ab45-7aa8-4856-a6fa-c6b3e55228fc admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:42.016694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:42.076609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad0f65-a080-477b-8a3f-e6bf2a09f25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:42.078030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad0f65-a080-477b-8a3f-e6bf2a09f25b admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:42.078458 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad0f65-a080-477b-8a3f-e6bf2a09f25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:42.078823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad0f65-a080-477b-8a3f-e6bf2a09f25b admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:42.079586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad0f65-a080-477b-8a3f-e6bf2a09f25b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:42.082183 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad0f65-a080-477b-8a3f-e6bf2a09f25b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:42.097948 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1001/1989] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:57:42.100660 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b89c56-a2d8-4118-841a-22504c7de94b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:42.114193 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b89c56-a2d8-4118-841a-22504c7de94b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:42.129668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd009f8-da01-464c-9a91-6ba4c02121f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:42.130446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd009f8-da01-464c-9a91-6ba4c02121f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:42.130764 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd009f8-da01-464c-9a91-6ba4c02121f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:42.131309 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd009f8-da01-464c-9a91-6ba4c02121f2 admin admin] PATH_INFO: `/v3/projects/3bc6c32abb8142c98896ccbd7f634718/users/92edc0064fb6489d847b4d1206eccdfa/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:42.133522 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd009f8-da01-464c-9a91-6ba4c02121f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc6c32abb8142c98896ccbd7f634718, user_id=92edc0064fb6489d847b4d1206eccdfa, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:42.137395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd009f8-da01-464c-9a91-6ba4c02121f2 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:42.154144 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 989/1990] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:42 2022] PUT /identity/v3/projects/3bc6c32abb8142c98896ccbd7f634718/users/92edc0064fb6489d847b4d1206eccdfa/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:42.167015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949bad9-421b-4287-8613-714f0849d111 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:42.167015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949bad9-421b-4287-8613-714f0849d111 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:42.167015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949bad9-421b-4287-8613-714f0849d111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:42.211062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9949bad9-421b-4287-8613-714f0849d111 None None] MFA Rules not processed for user `92edc0064fb6489d847b4d1206eccdfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:42.212400 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9949bad9-421b-4287-8613-714f0849d111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:42.299891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1002/1991] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:42.313839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd009f8-da01-464c-9a91-6ba4c02121f2 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:42.315613 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-abd009f8-da01-464c-9a91-6ba4c02121f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:42.394601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd009f8-da01-464c-9a91-6ba4c02121f2 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:42.399006 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076c88b-96ef-4201-9f42-2c74036438ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:42.399786 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076c88b-96ef-4201-9f42-2c74036438ac service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:42.400067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076c88b-96ef-4201-9f42-2c74036438ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:42.400433 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076c88b-96ef-4201-9f42-2c74036438ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:42.416253 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:42.478158 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076c88b-96ef-4201-9f42-2c74036438ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:42.479639 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076c88b-96ef-4201-9f42-2c74036438ac service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:42.490698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 990/1992] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:57:44.078348 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949bad9-421b-4287-8613-714f0849d111 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:44.089256 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949bad9-421b-4287-8613-714f0849d111 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:44.092852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bf0f8-82d3-44c2-be99-4f8b13a525cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:44.093467 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bf0f8-82d3-44c2-be99-4f8b13a525cd service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:44.093738 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bf0f8-82d3-44c2-be99-4f8b13a525cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:44.094001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bf0f8-82d3-44c2-be99-4f8b13a525cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:44.156086 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4bf0f8-82d3-44c2-be99-4f8b13a525cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:44.157565 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4bf0f8-82d3-44c2-be99-4f8b13a525cd service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:44.168950 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1003/1993] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:44.882673 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076c88b-96ef-4201-9f42-2c74036438ac service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:44.889452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076c88b-96ef-4201-9f42-2c74036438ac service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:44.894692 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b692d6-815f-4ef9-afcb-d131bb5ba4b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:44.895405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b692d6-815f-4ef9-afcb-d131bb5ba4b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:44.895750 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b692d6-815f-4ef9-afcb-d131bb5ba4b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:44.896043 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b692d6-815f-4ef9-afcb-d131bb5ba4b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:44.907765 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b692d6-815f-4ef9-afcb-d131bb5ba4b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:44.909271 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b692d6-815f-4ef9-afcb-d131bb5ba4b2 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:44.926449 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 991/1994] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:52.168254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bf0f8-82d3-44c2-be99-4f8b13a525cd service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:52.227765 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:52.244318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bf0f8-82d3-44c2-be99-4f8b13a525cd service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:52.250391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c590db8-789d-4444-b5f0-7a90c0b37144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:52.251044 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c590db8-789d-4444-b5f0-7a90c0b37144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:52.251323 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c590db8-789d-4444-b5f0-7a90c0b37144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:52.251598 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c590db8-789d-4444-b5f0-7a90c0b37144 admin admin] PATH_INFO: `/v3/users/52a638e8e76a4bcf8b5c5ac8cc1f5efe` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:52.252745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c590db8-789d-4444-b5f0-7a90c0b37144 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52a638e8e76a4bcf8b5c5ac8cc1f5efe)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:52.254693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c590db8-789d-4444-b5f0-7a90c0b37144 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:52.315934 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0c590db8-789d-4444-b5f0-7a90c0b37144 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52a638e8e76a4bcf8b5c5ac8cc1f5efe'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:52.316623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0c590db8-789d-4444-b5f0-7a90c0b37144 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52a638e8e76a4bcf8b5c5ac8cc1f5efe'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:52.325577 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0c590db8-789d-4444-b5f0-7a90c0b37144 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52a638e8e76a4bcf8b5c5ac8cc1f5efe'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:52.333181 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0c590db8-789d-4444-b5f0-7a90c0b37144 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52a638e8e76a4bcf8b5c5ac8cc1f5efe'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:52.348491 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0c590db8-789d-4444-b5f0-7a90c0b37144 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52a638e8e76a4bcf8b5c5ac8cc1f5efe'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:52.355658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1004/1995] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:52 2022] DELETE /identity/v3/users/52a638e8e76a4bcf8b5c5ac8cc1f5efe => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:52.360717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e305cd-34f7-4e55-8a3e-be49cc0ada8b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:52.361497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e305cd-34f7-4e55-8a3e-be49cc0ada8b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:52.361942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e305cd-34f7-4e55-8a3e-be49cc0ada8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:52.390226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-45e305cd-34f7-4e55-8a3e-be49cc0ada8b None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:52.391470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-45e305cd-34f7-4e55-8a3e-be49cc0ada8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:52.484730 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 992/1996] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:52.492502 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c590db8-789d-4444-b5f0-7a90c0b37144 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:52.494081 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0c590db8-789d-4444-b5f0-7a90c0b37144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:52.572361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c590db8-789d-4444-b5f0-7a90c0b37144 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:52.576187 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553cf96-1815-4eb2-bcba-4827d67d438f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:52.576824 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553cf96-1815-4eb2-bcba-4827d67d438f service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:52.577104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553cf96-1815-4eb2-bcba-4827d67d438f service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:52.577376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553cf96-1815-4eb2-bcba-4827d67d438f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:52.589257 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553cf96-1815-4eb2-bcba-4827d67d438f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:52.591209 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553cf96-1815-4eb2-bcba-4827d67d438f service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:52.605242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1005/1997] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:53.616887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e305cd-34f7-4e55-8a3e-be49cc0ada8b None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:53.619777 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-45e305cd-34f7-4e55-8a3e-be49cc0ada8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:53.638774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e305cd-34f7-4e55-8a3e-be49cc0ada8b None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:53.647465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678e2df-ed99-4f16-8079-f1bc5c91a669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:53.649137 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678e2df-ed99-4f16-8079-f1bc5c91a669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:53.650204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678e2df-ed99-4f16-8079-f1bc5c91a669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:53.650739 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678e2df-ed99-4f16-8079-f1bc5c91a669 admin admin] PATH_INFO: `/v3/projects/d473cb5f9a61464c8bc8034d24bdf1b5` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:53.652468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678e2df-ed99-4f16-8079-f1bc5c91a669 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d473cb5f9a61464c8bc8034d24bdf1b5)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:53.671352 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678e2df-ed99-4f16-8079-f1bc5c91a669 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:53.748029 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8678e2df-ed99-4f16-8079-f1bc5c91a669 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd473cb5f9a61464c8bc8034d24bdf1b5', 'request_id': 'req-8678e2df-ed99-4f16-8079-f1bc5c91a669'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:53.765039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8678e2df-ed99-4f16-8079-f1bc5c91a669 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd473cb5f9a61464c8bc8034d24bdf1b5', 'request_id': 'req-8678e2df-ed99-4f16-8079-f1bc5c91a669'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:53.773157 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8678e2df-ed99-4f16-8079-f1bc5c91a669 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd473cb5f9a61464c8bc8034d24bdf1b5', 'request_id': 'req-8678e2df-ed99-4f16-8079-f1bc5c91a669'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:53.808300 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8678e2df-ed99-4f16-8079-f1bc5c91a669 admin admin] The token cache is being invalidate because project d473cb5f9a61464c8bc8034d24bdf1b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:53.808300 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8678e2df-ed99-4f16-8079-f1bc5c91a669 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:53.808300 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-8678e2df-ed99-4f16-8079-f1bc5c91a669 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:53.810238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 993/1998] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:53 2022] DELETE /identity/v3/projects/d473cb5f9a61464c8bc8034d24bdf1b5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:53.815997 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553cf96-1815-4eb2-bcba-4827d67d438f service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:53.835048 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553cf96-1815-4eb2-bcba-4827d67d438f service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:53.923947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af484b-de10-4402-a39f-e8f37c1f153e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:53.924703 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af484b-de10-4402-a39f-e8f37c1f153e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:53.925048 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af484b-de10-4402-a39f-e8f37c1f153e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:53.925343 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af484b-de10-4402-a39f-e8f37c1f153e admin admin] PATH_INFO: `/v3/users/56b80e932a264d028a1018c7cfdeed46` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:53.926543 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af484b-de10-4402-a39f-e8f37c1f153e admin admin] RBAC: Authorizing `identity:delete_user(user_id=56b80e932a264d028a1018c7cfdeed46)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:53.928824 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af484b-de10-4402-a39f-e8f37c1f153e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:53.944669 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:53.995857 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-42af484b-de10-4402-a39f-e8f37c1f153e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56b80e932a264d028a1018c7cfdeed46'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:53.996648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-42af484b-de10-4402-a39f-e8f37c1f153e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56b80e932a264d028a1018c7cfdeed46'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.007571 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-42af484b-de10-4402-a39f-e8f37c1f153e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56b80e932a264d028a1018c7cfdeed46'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.018389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-42af484b-de10-4402-a39f-e8f37c1f153e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56b80e932a264d028a1018c7cfdeed46'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.039055 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-42af484b-de10-4402-a39f-e8f37c1f153e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56b80e932a264d028a1018c7cfdeed46'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.047686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1006/1999] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:53 2022] DELETE /identity/v3/users/56b80e932a264d028a1018c7cfdeed46 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:54.381697 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678e2df-ed99-4f16-8079-f1bc5c91a669 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:54.383408 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8678e2df-ed99-4f16-8079-f1bc5c91a669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:54.403273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678e2df-ed99-4f16-8079-f1bc5c91a669 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:54.427877 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:54.477179 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b31c89-b243-4676-85e4-0fd0485ee2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:54.477838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b31c89-b243-4676-85e4-0fd0485ee2d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:54.478121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b31c89-b243-4676-85e4-0fd0485ee2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:54.478397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b31c89-b243-4676-85e4-0fd0485ee2d0 admin admin] PATH_INFO: `/v3/projects/6368a76730304cd5a71b863a7d5f86e0` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:54.479575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b31c89-b243-4676-85e4-0fd0485ee2d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6368a76730304cd5a71b863a7d5f86e0)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:54.481578 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b31c89-b243-4676-85e4-0fd0485ee2d0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:54.540984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d4b31c89-b243-4676-85e4-0fd0485ee2d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6368a76730304cd5a71b863a7d5f86e0', 'request_id': 'req-d4b31c89-b243-4676-85e4-0fd0485ee2d0'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.558133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d4b31c89-b243-4676-85e4-0fd0485ee2d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6368a76730304cd5a71b863a7d5f86e0', 'request_id': 'req-d4b31c89-b243-4676-85e4-0fd0485ee2d0'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.565780 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d4b31c89-b243-4676-85e4-0fd0485ee2d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6368a76730304cd5a71b863a7d5f86e0', 'request_id': 'req-d4b31c89-b243-4676-85e4-0fd0485ee2d0'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.582711 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d4b31c89-b243-4676-85e4-0fd0485ee2d0 admin admin] The token cache is being invalidate because project 6368a76730304cd5a71b863a7d5f86e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:54.583204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d4b31c89-b243-4676-85e4-0fd0485ee2d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.583924 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d4b31c89-b243-4676-85e4-0fd0485ee2d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.586678 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 994/2000] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:54 2022] DELETE /identity/v3/projects/6368a76730304cd5a71b863a7d5f86e0 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:54.591254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af484b-de10-4402-a39f-e8f37c1f153e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:54.593011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-42af484b-de10-4402-a39f-e8f37c1f153e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:54.607401 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af484b-de10-4402-a39f-e8f37c1f153e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:54.667274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de76f7-7985-495d-a21c-ff752504e51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:54.667930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de76f7-7985-495d-a21c-ff752504e51f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:54.668243 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de76f7-7985-495d-a21c-ff752504e51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:54.668616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de76f7-7985-495d-a21c-ff752504e51f admin admin] PATH_INFO: `/v3/users/92edc0064fb6489d847b4d1206eccdfa` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:54.670015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de76f7-7985-495d-a21c-ff752504e51f admin admin] RBAC: Authorizing `identity:delete_user(user_id=92edc0064fb6489d847b4d1206eccdfa)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:54.672066 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de76f7-7985-495d-a21c-ff752504e51f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:54.736910 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b31c89-b243-4676-85e4-0fd0485ee2d0 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:54.741345 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81de76f7-7985-495d-a21c-ff752504e51f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92edc0064fb6489d847b4d1206eccdfa'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.742086 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d4b31c89-b243-4676-85e4-0fd0485ee2d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:54.743084 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81de76f7-7985-495d-a21c-ff752504e51f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92edc0064fb6489d847b4d1206eccdfa'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.761051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81de76f7-7985-495d-a21c-ff752504e51f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92edc0064fb6489d847b4d1206eccdfa'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.766181 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b31c89-b243-4676-85e4-0fd0485ee2d0 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:54.771409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824dfaa-f1d4-4e27-ba4f-27ae7f92855f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:54.772849 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824dfaa-f1d4-4e27-ba4f-27ae7f92855f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:54.773205 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824dfaa-f1d4-4e27-ba4f-27ae7f92855f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:54.773544 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824dfaa-f1d4-4e27-ba4f-27ae7f92855f admin admin] PATH_INFO: `/v3/users/80ba72a7414241bf829be747dd6ab276` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:54.775013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824dfaa-f1d4-4e27-ba4f-27ae7f92855f admin admin] RBAC: Authorizing `identity:delete_user(user_id=80ba72a7414241bf829be747dd6ab276)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:54.777504 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824dfaa-f1d4-4e27-ba4f-27ae7f92855f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:54.779259 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81de76f7-7985-495d-a21c-ff752504e51f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92edc0064fb6489d847b4d1206eccdfa'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.794075 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81de76f7-7985-495d-a21c-ff752504e51f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92edc0064fb6489d847b4d1206eccdfa'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.800307 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1007/2001] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:54 2022] DELETE /identity/v3/users/92edc0064fb6489d847b4d1206eccdfa => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:54.835594 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6824dfaa-f1d4-4e27-ba4f-27ae7f92855f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80ba72a7414241bf829be747dd6ab276'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.846076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6824dfaa-f1d4-4e27-ba4f-27ae7f92855f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80ba72a7414241bf829be747dd6ab276'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.863704 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6824dfaa-f1d4-4e27-ba4f-27ae7f92855f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80ba72a7414241bf829be747dd6ab276'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.873622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6824dfaa-f1d4-4e27-ba4f-27ae7f92855f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80ba72a7414241bf829be747dd6ab276'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.875190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6824dfaa-f1d4-4e27-ba4f-27ae7f92855f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80ba72a7414241bf829be747dd6ab276'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:54.884140 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 995/2002] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:54 2022] DELETE /identity/v3/users/80ba72a7414241bf829be747dd6ab276 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:57:54.888131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8215a73-a77e-471a-93d0-02fc2e88bbc1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:54.888131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8215a73-a77e-471a-93d0-02fc2e88bbc1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:54.888131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8215a73-a77e-471a-93d0-02fc2e88bbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:54.902295 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:54.910393 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a8215a73-a77e-471a-93d0-02fc2e88bbc1 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:54.912284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a8215a73-a77e-471a-93d0-02fc2e88bbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:55.016469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1008/2003] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:55.039982 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824dfaa-f1d4-4e27-ba4f-27ae7f92855f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:55.041486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6824dfaa-f1d4-4e27-ba4f-27ae7f92855f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:55.088538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:55.156981 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824dfaa-f1d4-4e27-ba4f-27ae7f92855f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:55.162928 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be7227-a59e-4915-a243-b2e711e20b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:55.165173 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be7227-a59e-4915-a243-b2e711e20b22 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:55.165748 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be7227-a59e-4915-a243-b2e711e20b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:55.166316 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be7227-a59e-4915-a243-b2e711e20b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:55.180995 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be7227-a59e-4915-a243-b2e711e20b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:55.183039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be7227-a59e-4915-a243-b2e711e20b22 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:55.204298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 996/2004] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:55.432268 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8215a73-a77e-471a-93d0-02fc2e88bbc1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:55.433939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a8215a73-a77e-471a-93d0-02fc2e88bbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:55.470423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8215a73-a77e-471a-93d0-02fc2e88bbc1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:55.481215 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd6bca-e384-46ce-9c64-3d4772ca2c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:55.482759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd6bca-e384-46ce-9c64-3d4772ca2c84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:55.484040 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd6bca-e384-46ce-9c64-3d4772ca2c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:55.484689 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd6bca-e384-46ce-9c64-3d4772ca2c84 admin admin] PATH_INFO: `/v3/projects/3bc6c32abb8142c98896ccbd7f634718` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:55.487987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd6bca-e384-46ce-9c64-3d4772ca2c84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bc6c32abb8142c98896ccbd7f634718)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:55.497469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd6bca-e384-46ce-9c64-3d4772ca2c84 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:55.578170 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a8cd6bca-e384-46ce-9c64-3d4772ca2c84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bc6c32abb8142c98896ccbd7f634718', 'request_id': 'req-a8cd6bca-e384-46ce-9c64-3d4772ca2c84'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:55.591124 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a8cd6bca-e384-46ce-9c64-3d4772ca2c84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bc6c32abb8142c98896ccbd7f634718', 'request_id': 'req-a8cd6bca-e384-46ce-9c64-3d4772ca2c84'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:55.603152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a8cd6bca-e384-46ce-9c64-3d4772ca2c84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bc6c32abb8142c98896ccbd7f634718', 'request_id': 'req-a8cd6bca-e384-46ce-9c64-3d4772ca2c84'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:55.619031 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a8cd6bca-e384-46ce-9c64-3d4772ca2c84 admin admin] The token cache is being invalidate because project 3bc6c32abb8142c98896ccbd7f634718 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:55.619416 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a8cd6bca-e384-46ce-9c64-3d4772ca2c84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:55.620335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a8cd6bca-e384-46ce-9c64-3d4772ca2c84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:55.633168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1009/2005] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:55 2022] DELETE /identity/v3/projects/3bc6c32abb8142c98896ccbd7f634718 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:55.635455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be7227-a59e-4915-a243-b2e711e20b22 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:55.658631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-19be7227-a59e-4915-a243-b2e711e20b22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:55.692439 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be7227-a59e-4915-a243-b2e711e20b22 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:55.788923 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699b0e9-7e2a-41cf-a331-2974d77f6fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:55.790639 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699b0e9-7e2a-41cf-a331-2974d77f6fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:55.790922 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699b0e9-7e2a-41cf-a331-2974d77f6fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:55.791075 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699b0e9-7e2a-41cf-a331-2974d77f6fb9 admin admin] PATH_INFO: `/v3/projects/ac325a2b980843d2ba76fa295fe7293f` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:55.792653 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699b0e9-7e2a-41cf-a331-2974d77f6fb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac325a2b980843d2ba76fa295fe7293f)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:55.796150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699b0e9-7e2a-41cf-a331-2974d77f6fb9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:55.869393 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c699b0e9-7e2a-41cf-a331-2974d77f6fb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac325a2b980843d2ba76fa295fe7293f', 'request_id': 'req-c699b0e9-7e2a-41cf-a331-2974d77f6fb9'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:55.882915 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c699b0e9-7e2a-41cf-a331-2974d77f6fb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac325a2b980843d2ba76fa295fe7293f', 'request_id': 'req-c699b0e9-7e2a-41cf-a331-2974d77f6fb9'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:55.890508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c699b0e9-7e2a-41cf-a331-2974d77f6fb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac325a2b980843d2ba76fa295fe7293f', 'request_id': 'req-c699b0e9-7e2a-41cf-a331-2974d77f6fb9'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:55.905122 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c699b0e9-7e2a-41cf-a331-2974d77f6fb9 admin admin] The token cache is being invalidate because project ac325a2b980843d2ba76fa295fe7293f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:57:55.905544 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c699b0e9-7e2a-41cf-a331-2974d77f6fb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:55.906395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c699b0e9-7e2a-41cf-a331-2974d77f6fb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:55.908985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 997/2006] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:57:55 2022] DELETE /identity/v3/projects/ac325a2b980843d2ba76fa295fe7293f => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:55.918991 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefe193-c928-44fe-8e1f-97d5ee772880 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:55.919916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefe193-c928-44fe-8e1f-97d5ee772880 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:55.920490 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefe193-c928-44fe-8e1f-97d5ee772880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:55.945904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ebefe193-c928-44fe-8e1f-97d5ee772880 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:55.947132 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ebefe193-c928-44fe-8e1f-97d5ee772880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:56.013449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1010/2007] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:57:56.023459 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1cb11-68fe-4b2a-9477-ed44017ad1d9 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:56.023762 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1cb11-68fe-4b2a-9477-ed44017ad1d9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:56.024031 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1cb11-68fe-4b2a-9477-ed44017ad1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:56.045390 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9ca1cb11-68fe-4b2a-9477-ed44017ad1d9 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:56.046619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9ca1cb11-68fe-4b2a-9477-ed44017ad1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:56.055498 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 998/2008] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:56.060357 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefe193-c928-44fe-8e1f-97d5ee772880 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:56.070559 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefe193-c928-44fe-8e1f-97d5ee772880 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:56.075320 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ac53c-efbc-4845-b88e-40fa2a48ae2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:56.075952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ac53c-efbc-4845-b88e-40fa2a48ae2e admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:56.076492 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ac53c-efbc-4845-b88e-40fa2a48ae2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:56.076971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ac53c-efbc-4845-b88e-40fa2a48ae2e admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:56.077836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ac53c-efbc-4845-b88e-40fa2a48ae2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:57:56.078140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ac53c-efbc-4845-b88e-40fa2a48ae2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:56.079809 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ac53c-efbc-4845-b88e-40fa2a48ae2e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:56.093101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1011/2009] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:57:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:57:56.098188 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1cb11-68fe-4b2a-9477-ed44017ad1d9 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:56.108814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1cb11-68fe-4b2a-9477-ed44017ad1d9 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:56.113907 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057d974-565f-4d11-a90a-b77548e13c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:56.114692 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057d974-565f-4d11-a90a-b77548e13c53 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:56.114973 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057d974-565f-4d11-a90a-b77548e13c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:56.115241 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057d974-565f-4d11-a90a-b77548e13c53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:56.116257 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057d974-565f-4d11-a90a-b77548e13c53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:56.118197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057d974-565f-4d11-a90a-b77548e13c53 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:56.151914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 999/2010] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:57:56 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:56.158452 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ac53c-efbc-4845-b88e-40fa2a48ae2e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:56.169390 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ac53c-efbc-4845-b88e-40fa2a48ae2e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:56.183682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:56.236543 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1ebf7-a8b9-4c74-9201-34b9642db16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:56.237307 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1ebf7-a8b9-4c74-9201-34b9642db16b admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:56.237587 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1ebf7-a8b9-4c74-9201-34b9642db16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:56.237859 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1ebf7-a8b9-4c74-9201-34b9642db16b admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:56.238502 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1ebf7-a8b9-4c74-9201-34b9642db16b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:56.241070 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1ebf7-a8b9-4c74-9201-34b9642db16b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:56.269554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1012/2011] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:57:56 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:57:56.270788 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057d974-565f-4d11-a90a-b77548e13c53 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:56.282976 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057d974-565f-4d11-a90a-b77548e13c53 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:56.289548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafb3a4-7855-4e47-ae69-eb03b40aada9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:56.290355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafb3a4-7855-4e47-ae69-eb03b40aada9 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:56.290638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafb3a4-7855-4e47-ae69-eb03b40aada9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:56.290915 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafb3a4-7855-4e47-ae69-eb03b40aada9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:56.291583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafb3a4-7855-4e47-ae69-eb03b40aada9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:56.293473 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafb3a4-7855-4e47-ae69-eb03b40aada9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:56.308161 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1000/2012] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:56.311561 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1ebf7-a8b9-4c74-9201-34b9642db16b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:56.322994 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1ebf7-a8b9-4c74-9201-34b9642db16b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:56.327946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938d062-4b71-46cb-ad30-89f1953d3232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:56.328608 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938d062-4b71-46cb-ad30-89f1953d3232 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:56.328890 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938d062-4b71-46cb-ad30-89f1953d3232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:56.329255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938d062-4b71-46cb-ad30-89f1953d3232 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:56.329848 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938d062-4b71-46cb-ad30-89f1953d3232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:56.331492 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938d062-4b71-46cb-ad30-89f1953d3232 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:56.344153 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1013/2013] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:57:56.347391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafb3a4-7855-4e47-ae69-eb03b40aada9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:56.358695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafb3a4-7855-4e47-ae69-eb03b40aada9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:56.363856 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc058b6-afaf-4329-b15c-1d2868c5cdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:56.364802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc058b6-afaf-4329-b15c-1d2868c5cdfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:56.365343 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc058b6-afaf-4329-b15c-1d2868c5cdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:56.366324 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc058b6-afaf-4329-b15c-1d2868c5cdfb admin admin] PATH_INFO: `/v3/projects/2b4d6985fe004c19b1043d05373c33e9/users/6ad3b4d943af47e4b94518d514a4eccb/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:56.385033 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc058b6-afaf-4329-b15c-1d2868c5cdfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b4d6985fe004c19b1043d05373c33e9, user_id=6ad3b4d943af47e4b94518d514a4eccb, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:56.387856 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc058b6-afaf-4329-b15c-1d2868c5cdfb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:56.390437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:56.405876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1001/2014] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:57:56 2022] PUT /identity/v3/projects/2b4d6985fe004c19b1043d05373c33e9/users/6ad3b4d943af47e4b94518d514a4eccb/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:56.418295 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167f7c6-add1-4dd7-bb2a-55fc2b947dec None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:56.418536 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167f7c6-add1-4dd7-bb2a-55fc2b947dec None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:56.418645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167f7c6-add1-4dd7-bb2a-55fc2b947dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:56.454856 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e167f7c6-add1-4dd7-bb2a-55fc2b947dec None None] MFA Rules not processed for user `6ad3b4d943af47e4b94518d514a4eccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:56.456078 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e167f7c6-add1-4dd7-bb2a-55fc2b947dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:56.533886 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1014/2015] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:56.545673 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc058b6-afaf-4329-b15c-1d2868c5cdfb admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:56.547440 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-cfc058b6-afaf-4329-b15c-1d2868c5cdfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:56.608558 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc058b6-afaf-4329-b15c-1d2868c5cdfb admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:56.612213 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff0204-83be-4340-876d-263e30463e73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:56.612830 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff0204-83be-4340-876d-263e30463e73 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:56.613109 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff0204-83be-4340-876d-263e30463e73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:56.613378 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff0204-83be-4340-876d-263e30463e73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:56.623522 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff0204-83be-4340-876d-263e30463e73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:56.625015 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff0204-83be-4340-876d-263e30463e73 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:56.636469 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1002/2016] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:57:59.467421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167f7c6-add1-4dd7-bb2a-55fc2b947dec None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:59.552255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167f7c6-add1-4dd7-bb2a-55fc2b947dec None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:59.557498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85edfe8-a2e5-4b79-b449-ca5d4235713c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:59.558351 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85edfe8-a2e5-4b79-b449-ca5d4235713c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.558704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85edfe8-a2e5-4b79-b449-ca5d4235713c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.559054 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85edfe8-a2e5-4b79-b449-ca5d4235713c admin admin] PATH_INFO: `/v3/users/6ad3b4d943af47e4b94518d514a4eccb` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.560662 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85edfe8-a2e5-4b79-b449-ca5d4235713c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ad3b4d943af47e4b94518d514a4eccb)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:59.562726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85edfe8-a2e5-4b79-b449-ca5d4235713c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:59.601414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:57:59.628507 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d85edfe8-a2e5-4b79-b449-ca5d4235713c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ad3b4d943af47e4b94518d514a4eccb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:59.629193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d85edfe8-a2e5-4b79-b449-ca5d4235713c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ad3b4d943af47e4b94518d514a4eccb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:59.641548 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d85edfe8-a2e5-4b79-b449-ca5d4235713c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ad3b4d943af47e4b94518d514a4eccb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:59.651131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d85edfe8-a2e5-4b79-b449-ca5d4235713c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ad3b4d943af47e4b94518d514a4eccb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:59.666250 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d85edfe8-a2e5-4b79-b449-ca5d4235713c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ad3b4d943af47e4b94518d514a4eccb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:57:59.673684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1015/2017] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:57:59 2022] DELETE /identity/v3/users/6ad3b4d943af47e4b94518d514a4eccb => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:57:59.680963 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168dce4a-957a-43e6-bb86-66437ef31430 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.681466 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168dce4a-957a-43e6-bb86-66437ef31430 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.681971 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168dce4a-957a-43e6-bb86-66437ef31430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.705095 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-168dce4a-957a-43e6-bb86-66437ef31430 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:57:59.706485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-168dce4a-957a-43e6-bb86-66437ef31430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:59.797660 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1003/2018] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:57:59.809411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85edfe8-a2e5-4b79-b449-ca5d4235713c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:57:59.811401 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d85edfe8-a2e5-4b79-b449-ca5d4235713c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:57:59.887055 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85edfe8-a2e5-4b79-b449-ca5d4235713c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:57:59.891243 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e388ca-0156-4f6d-8f60-9c6dfc3b44eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:57:59.892265 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e388ca-0156-4f6d-8f60-9c6dfc3b44eb service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:57:59.892627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e388ca-0156-4f6d-8f60-9c6dfc3b44eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:57:59.892906 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e388ca-0156-4f6d-8f60-9c6dfc3b44eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:57:59.904370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e388ca-0156-4f6d-8f60-9c6dfc3b44eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:57:59.905828 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e388ca-0156-4f6d-8f60-9c6dfc3b44eb service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:57:59.921733 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1016/2019] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:57:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:00.244717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168dce4a-957a-43e6-bb86-66437ef31430 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:00.246719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-168dce4a-957a-43e6-bb86-66437ef31430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:00.262192 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168dce4a-957a-43e6-bb86-66437ef31430 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.267407 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394fc77-b88c-481b-813b-25fe319cb5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.268086 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394fc77-b88c-481b-813b-25fe319cb5dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.268396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394fc77-b88c-481b-813b-25fe319cb5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.268673 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394fc77-b88c-481b-813b-25fe319cb5dd admin admin] PATH_INFO: `/v3/projects/2b4d6985fe004c19b1043d05373c33e9` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.269878 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394fc77-b88c-481b-813b-25fe319cb5dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b4d6985fe004c19b1043d05373c33e9)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.271802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394fc77-b88c-481b-813b-25fe319cb5dd admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:00.362037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b394fc77-b88c-481b-813b-25fe319cb5dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b4d6985fe004c19b1043d05373c33e9', 'request_id': 'req-b394fc77-b88c-481b-813b-25fe319cb5dd'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:00.386008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b394fc77-b88c-481b-813b-25fe319cb5dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b4d6985fe004c19b1043d05373c33e9', 'request_id': 'req-b394fc77-b88c-481b-813b-25fe319cb5dd'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:00.402386 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b394fc77-b88c-481b-813b-25fe319cb5dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b4d6985fe004c19b1043d05373c33e9', 'request_id': 'req-b394fc77-b88c-481b-813b-25fe319cb5dd'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:00.417594 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b394fc77-b88c-481b-813b-25fe319cb5dd admin admin] The token cache is being invalidate because project 2b4d6985fe004c19b1043d05373c33e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:00.418883 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b394fc77-b88c-481b-813b-25fe319cb5dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:00.419565 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b394fc77-b88c-481b-813b-25fe319cb5dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:00.421593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1004/2020] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:00 2022] DELETE /identity/v3/projects/2b4d6985fe004c19b1043d05373c33e9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:00.426176 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b12ff4-6437-4dea-9dea-46d11f22367f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.426598 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b12ff4-6437-4dea-9dea-46d11f22367f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.426958 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b12ff4-6437-4dea-9dea-46d11f22367f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.452113 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f0b12ff4-6437-4dea-9dea-46d11f22367f None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:00.453440 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f0b12ff4-6437-4dea-9dea-46d11f22367f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:00.526158 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1017/2021] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:00.534031 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f80d0b-e260-4d72-830c-b0f8e303c95f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.534371 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f80d0b-e260-4d72-830c-b0f8e303c95f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.534651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f80d0b-e260-4d72-830c-b0f8e303c95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.555360 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b3f80d0b-e260-4d72-830c-b0f8e303c95f None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:00.556623 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b3f80d0b-e260-4d72-830c-b0f8e303c95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:00.564926 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1005/2022] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:00.570357 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b12ff4-6437-4dea-9dea-46d11f22367f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:00.580771 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b12ff4-6437-4dea-9dea-46d11f22367f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.587535 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7378a21-eccd-49c4-b084-1cc8496bca8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.588186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7378a21-eccd-49c4-b084-1cc8496bca8c admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.588462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7378a21-eccd-49c4-b084-1cc8496bca8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.588735 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7378a21-eccd-49c4-b084-1cc8496bca8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.589387 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7378a21-eccd-49c4-b084-1cc8496bca8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:00.589691 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7378a21-eccd-49c4-b084-1cc8496bca8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.591337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7378a21-eccd-49c4-b084-1cc8496bca8c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:00.605958 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1018/2023] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:58:00.611580 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f80d0b-e260-4d72-830c-b0f8e303c95f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:00.623910 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f80d0b-e260-4d72-830c-b0f8e303c95f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.628631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f61f3-edf8-419e-8f8b-eaa70acf46c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.629243 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f61f3-edf8-419e-8f8b-eaa70acf46c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.629518 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f61f3-edf8-419e-8f8b-eaa70acf46c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.629785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f61f3-edf8-419e-8f8b-eaa70acf46c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.630410 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f61f3-edf8-419e-8f8b-eaa70acf46c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.632782 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f61f3-edf8-419e-8f8b-eaa70acf46c7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:00.646249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:00.674725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1006/2024] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:00 2022] POST /identity/v3/projects => generated 369 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:00.676015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7378a21-eccd-49c4-b084-1cc8496bca8c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:00.700940 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7378a21-eccd-49c4-b084-1cc8496bca8c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.775641 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:00.820281 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482d833-e94c-4229-b351-ed3c609c2141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.820948 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482d833-e94c-4229-b351-ed3c609c2141 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.821352 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482d833-e94c-4229-b351-ed3c609c2141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.821693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482d833-e94c-4229-b351-ed3c609c2141 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.822495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482d833-e94c-4229-b351-ed3c609c2141 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.827610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482d833-e94c-4229-b351-ed3c609c2141 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:00.869670 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1019/2025] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:00 2022] POST /identity/v3/users => generated 285 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:00.874569 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f61f3-edf8-419e-8f8b-eaa70acf46c7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:00.890120 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f61f3-edf8-419e-8f8b-eaa70acf46c7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.903297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666a5b8-29ba-4787-babd-a22e0e48858a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.903691 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666a5b8-29ba-4787-babd-a22e0e48858a admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.903977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666a5b8-29ba-4787-babd-a22e0e48858a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.904417 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666a5b8-29ba-4787-babd-a22e0e48858a admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.905422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666a5b8-29ba-4787-babd-a22e0e48858a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.906762 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666a5b8-29ba-4787-babd-a22e0e48858a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:00.920727 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1007/2026] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:58:00.925477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482d833-e94c-4229-b351-ed3c609c2141 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:00.941434 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482d833-e94c-4229-b351-ed3c609c2141 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:00.963005 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842c177-d380-4fd2-8b24-343af6d94d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:00.964301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842c177-d380-4fd2-8b24-343af6d94d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:00.964797 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842c177-d380-4fd2-8b24-343af6d94d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:00.965214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842c177-d380-4fd2-8b24-343af6d94d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:00.966706 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842c177-d380-4fd2-8b24-343af6d94d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:00.969347 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842c177-d380-4fd2-8b24-343af6d94d45 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:01.000149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1020/2027] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:01.002248 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666a5b8-29ba-4787-babd-a22e0e48858a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:01.021063 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666a5b8-29ba-4787-babd-a22e0e48858a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:01.041308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240879f5-a738-4d23-bccd-90eb95eb0f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:01.042238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240879f5-a738-4d23-bccd-90eb95eb0f57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:01.042584 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240879f5-a738-4d23-bccd-90eb95eb0f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:01.042919 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240879f5-a738-4d23-bccd-90eb95eb0f57 admin admin] PATH_INFO: `/v3/projects/8b9640d286cd4b1c86958a4cedc3e1a0/users/0212144e820f4724ab36f683d6448794/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:01.061763 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240879f5-a738-4d23-bccd-90eb95eb0f57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b9640d286cd4b1c86958a4cedc3e1a0, user_id=0212144e820f4724ab36f683d6448794, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:01.066017 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240879f5-a738-4d23-bccd-90eb95eb0f57 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:01.093488 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1008/2028] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:01 2022] PUT /identity/v3/projects/8b9640d286cd4b1c86958a4cedc3e1a0/users/0212144e820f4724ab36f683d6448794/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:01.105563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fe04c-1aa0-42f4-a352-e300a20d9321 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:01.105856 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fe04c-1aa0-42f4-a352-e300a20d9321 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:01.106161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fe04c-1aa0-42f4-a352-e300a20d9321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:01.139417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-cd4fe04c-1aa0-42f4-a352-e300a20d9321 None None] MFA Rules not processed for user `0212144e820f4724ab36f683d6448794`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:01.140703 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cd4fe04c-1aa0-42f4-a352-e300a20d9321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:01.274273 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1021/2029] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:01.275175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240879f5-a738-4d23-bccd-90eb95eb0f57 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:01.277164 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-240879f5-a738-4d23-bccd-90eb95eb0f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:01.361773 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240879f5-a738-4d23-bccd-90eb95eb0f57 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:01.367282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899c5b2-df71-4a8d-86b9-966fd930c6e7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5979387f7aa54619b8d429368c306cc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:01.368220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899c5b2-df71-4a8d-86b9-966fd930c6e7 service glance] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:01.368736 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899c5b2-df71-4a8d-86b9-966fd930c6e7 service glance] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:01.369286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899c5b2-df71-4a8d-86b9-966fd930c6e7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:01.384891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7899c5b2-df71-4a8d-86b9-966fd930c6e7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:01.386825 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7899c5b2-df71-4a8d-86b9-966fd930c6e7 service glance] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:01.433151 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1009/2030] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:01.703818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c020b5-abe1-4fed-8db8-a20ebc520025 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:01.704384 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c020b5-abe1-4fed-8db8-a20ebc520025 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:01.704384 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c020b5-abe1-4fed-8db8-a20ebc520025 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:01.705801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1022/2031] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 24 08:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:58:01.722332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd9401-9745-4775-91f1-fac48b69ded9 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:01.722622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd9401-9745-4775-91f1-fac48b69ded9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:01.722936 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd9401-9745-4775-91f1-fac48b69ded9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:01.752948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-4bcd9401-9745-4775-91f1-fac48b69ded9 None None] MFA Rules not processed for user `33a0844a1c8f4ea4932e12d53e5ee9ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:01.754299 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4bcd9401-9745-4775-91f1-fac48b69ded9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:01.800728 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c020b5-abe1-4fed-8db8-a20ebc520025 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:01.802360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-80c020b5-abe1-4fed-8db8-a20ebc520025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:01.875974 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1010/2032] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 24 08:58:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:01.883541 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd9401-9745-4775-91f1-fac48b69ded9 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:01.885259 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4bcd9401-9745-4775-91f1-fac48b69ded9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:01.899973 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd9401-9745-4775-91f1-fac48b69ded9 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:01.905617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a071f55-1268-4698-92bf-351520c6273c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33a0844a1c8f4ea4932e12d53e5ee9ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:01.906454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a071f55-1268-4698-92bf-351520c6273c service placement] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:01.906942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a071f55-1268-4698-92bf-351520c6273c service placement] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:01.907436 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a071f55-1268-4698-92bf-351520c6273c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:01.913482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c020b5-abe1-4fed-8db8-a20ebc520025 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:01.919362 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a071f55-1268-4698-92bf-351520c6273c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:01.919631 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75f851-60f4-412d-99df-c4d198a7edd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:01.920536 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75f851-60f4-412d-99df-c4d198a7edd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:01.920889 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75f851-60f4-412d-99df-c4d198a7edd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:01.921252 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75f851-60f4-412d-99df-c4d198a7edd0 admin admin] PATH_INFO: `/v3/users/5c036cb4ba8e4a16bd9e884243e665c1` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:01.922168 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a071f55-1268-4698-92bf-351520c6273c service placement] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:01.923195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75f851-60f4-412d-99df-c4d198a7edd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c036cb4ba8e4a16bd9e884243e665c1)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:01.926213 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75f851-60f4-412d-99df-c4d198a7edd0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:01.950752 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1011/2033] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:02.024314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:02.026347 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7b75f851-60f4-412d-99df-c4d198a7edd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c036cb4ba8e4a16bd9e884243e665c1'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:02.027365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7b75f851-60f4-412d-99df-c4d198a7edd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c036cb4ba8e4a16bd9e884243e665c1'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:02.048969 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7b75f851-60f4-412d-99df-c4d198a7edd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c036cb4ba8e4a16bd9e884243e665c1'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:02.057742 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7b75f851-60f4-412d-99df-c4d198a7edd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c036cb4ba8e4a16bd9e884243e665c1'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:02.070881 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7b75f851-60f4-412d-99df-c4d198a7edd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c036cb4ba8e4a16bd9e884243e665c1'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:02.077711 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1023/2034] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:01 2022] DELETE /identity/v3/users/5c036cb4ba8e4a16bd9e884243e665c1 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:02.271702 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a071f55-1268-4698-92bf-351520c6273c service placement] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:02.283103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a071f55-1268-4698-92bf-351520c6273c service placement] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:02.333372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:02.352472 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac48e14-5cde-450a-b007-d2d9439c661c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:02.353105 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac48e14-5cde-450a-b007-d2d9439c661c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:02.353439 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac48e14-5cde-450a-b007-d2d9439c661c admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:02.353661 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac48e14-5cde-450a-b007-d2d9439c661c admin admin] PATH_INFO: `/v3/projects/39195166aed84c998b13b926c0a39ab7` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:02.354766 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac48e14-5cde-450a-b007-d2d9439c661c admin admin] RBAC: Authorizing `identity:delete_project(project_id=39195166aed84c998b13b926c0a39ab7)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:02.356719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac48e14-5cde-450a-b007-d2d9439c661c admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:02.417500 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2ac48e14-5cde-450a-b007-d2d9439c661c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39195166aed84c998b13b926c0a39ab7', 'request_id': 'req-2ac48e14-5cde-450a-b007-d2d9439c661c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:02.427873 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2ac48e14-5cde-450a-b007-d2d9439c661c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39195166aed84c998b13b926c0a39ab7', 'request_id': 'req-2ac48e14-5cde-450a-b007-d2d9439c661c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:02.435350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2ac48e14-5cde-450a-b007-d2d9439c661c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39195166aed84c998b13b926c0a39ab7', 'request_id': 'req-2ac48e14-5cde-450a-b007-d2d9439c661c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:02.451023 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2ac48e14-5cde-450a-b007-d2d9439c661c admin admin] The token cache is being invalidate because project 39195166aed84c998b13b926c0a39ab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:02.451298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2ac48e14-5cde-450a-b007-d2d9439c661c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:02.451955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2ac48e14-5cde-450a-b007-d2d9439c661c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:02.453620 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1012/2035] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:02 2022] DELETE /identity/v3/projects/39195166aed84c998b13b926c0a39ab7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:02.460023 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b45fd1-2648-4a5b-9999-364af5b847e7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:02.460023 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b45fd1-2648-4a5b-9999-364af5b847e7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:02.460263 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b45fd1-2648-4a5b-9999-364af5b847e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:02.485448 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e6b45fd1-2648-4a5b-9999-364af5b847e7 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:02.486720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e6b45fd1-2648-4a5b-9999-364af5b847e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:02.548734 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1024/2036] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:02.557356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26b05b-c7e8-4bd0-a21d-78a5abe02ad6 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:02.557638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26b05b-c7e8-4bd0-a21d-78a5abe02ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:02.557905 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26b05b-c7e8-4bd0-a21d-78a5abe02ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:02.577603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-dd26b05b-c7e8-4bd0-a21d-78a5abe02ad6 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:02.578806 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dd26b05b-c7e8-4bd0-a21d-78a5abe02ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:02.587573 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1013/2037] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:02.592373 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b45fd1-2648-4a5b-9999-364af5b847e7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:02.603933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b45fd1-2648-4a5b-9999-364af5b847e7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:02.608710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fa06d-3216-4672-9742-b6dfa9aa379e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:02.609313 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fa06d-3216-4672-9742-b6dfa9aa379e admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:02.609596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fa06d-3216-4672-9742-b6dfa9aa379e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:02.609866 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fa06d-3216-4672-9742-b6dfa9aa379e admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:02.610522 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662fa06d-3216-4672-9742-b6dfa9aa379e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:02.610912 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662fa06d-3216-4672-9742-b6dfa9aa379e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:02.612483 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662fa06d-3216-4672-9742-b6dfa9aa379e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:02.626977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1025/2038] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:58:02.631314 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26b05b-c7e8-4bd0-a21d-78a5abe02ad6 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:02.641866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26b05b-c7e8-4bd0-a21d-78a5abe02ad6 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:02.647400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7d4e6-10f6-4c05-980e-beb44feadb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:02.648123 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7d4e6-10f6-4c05-980e-beb44feadb04 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:02.648526 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7d4e6-10f6-4c05-980e-beb44feadb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:02.648909 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7d4e6-10f6-4c05-980e-beb44feadb04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:02.649642 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa7d4e6-10f6-4c05-980e-beb44feadb04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:02.652330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa7d4e6-10f6-4c05-980e-beb44feadb04 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:02.685417 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1014/2039] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:02 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:02.686540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fa06d-3216-4672-9742-b6dfa9aa379e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:02.697826 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fa06d-3216-4672-9742-b6dfa9aa379e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:02.763003 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c6017-38c6-4e1a-a700-12c8fd335889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:02.763663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c6017-38c6-4e1a-a700-12c8fd335889 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:02.763879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c6017-38c6-4e1a-a700-12c8fd335889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:02.764193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c6017-38c6-4e1a-a700-12c8fd335889 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:02.764827 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c6017-38c6-4e1a-a700-12c8fd335889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:02.766764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c6017-38c6-4e1a-a700-12c8fd335889 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:02.794637 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1026/2040] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:02 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:02.795858 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7d4e6-10f6-4c05-980e-beb44feadb04 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:02.807888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7d4e6-10f6-4c05-980e-beb44feadb04 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:02.812677 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969c0b0-6fcd-4ab3-a31e-13e7d7f7f4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:02.813498 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969c0b0-6fcd-4ab3-a31e-13e7d7f7f4b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:02.813749 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969c0b0-6fcd-4ab3-a31e-13e7d7f7f4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:02.814008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969c0b0-6fcd-4ab3-a31e-13e7d7f7f4b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:02.814685 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b969c0b0-6fcd-4ab3-a31e-13e7d7f7f4b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:02.816426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b969c0b0-6fcd-4ab3-a31e-13e7d7f7f4b1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:02.831246 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1015/2041] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:02.832466 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c6017-38c6-4e1a-a700-12c8fd335889 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:02.843440 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c6017-38c6-4e1a-a700-12c8fd335889 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:02.849808 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a9604-4a1d-40e2-a64b-14866ecbcfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:02.851042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a9604-4a1d-40e2-a64b-14866ecbcfad admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:02.851582 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a9604-4a1d-40e2-a64b-14866ecbcfad admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:02.851963 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a9604-4a1d-40e2-a64b-14866ecbcfad admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:02.853888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154a9604-4a1d-40e2-a64b-14866ecbcfad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:02.855526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154a9604-4a1d-40e2-a64b-14866ecbcfad admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:02.867719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1027/2042] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:02.872211 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969c0b0-6fcd-4ab3-a31e-13e7d7f7f4b1 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:02.883559 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969c0b0-6fcd-4ab3-a31e-13e7d7f7f4b1 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:02.888646 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cf147-4cb4-45fa-a9e1-83fc261238cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:02.889324 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cf147-4cb4-45fa-a9e1-83fc261238cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:02.889607 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cf147-4cb4-45fa-a9e1-83fc261238cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:02.889910 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cf147-4cb4-45fa-a9e1-83fc261238cb admin admin] PATH_INFO: `/v3/projects/15af91597a02408a9516533b5153b214/users/709e0a7585674796ad2d61b560bc7350/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:02.906266 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163cf147-4cb4-45fa-a9e1-83fc261238cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=15af91597a02408a9516533b5153b214, user_id=709e0a7585674796ad2d61b560bc7350, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:02.909015 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163cf147-4cb4-45fa-a9e1-83fc261238cb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:02.925445 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1016/2043] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:02 2022] PUT /identity/v3/projects/15af91597a02408a9516533b5153b214/users/709e0a7585674796ad2d61b560bc7350/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:02.935769 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2cdd9-e6f3-4d77-be7d-60b01b73193a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:02.936184 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2cdd9-e6f3-4d77-be7d-60b01b73193a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:02.936485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2cdd9-e6f3-4d77-be7d-60b01b73193a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:02.971713 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-45c2cdd9-e6f3-4d77-be7d-60b01b73193a None None] MFA Rules not processed for user `709e0a7585674796ad2d61b560bc7350`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:02.971713 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-45c2cdd9-e6f3-4d77-be7d-60b01b73193a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:02.994445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:03.057703 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1028/2044] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:03.062431 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cf147-4cb4-45fa-a9e1-83fc261238cb admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:03.144159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cf147-4cb4-45fa-a9e1-83fc261238cb admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:03.150545 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb22fb-79c0-47b0-b7fd-a4615248fd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:03.151355 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb22fb-79c0-47b0-b7fd-a4615248fd03 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:03.151712 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb22fb-79c0-47b0-b7fd-a4615248fd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:03.152054 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb22fb-79c0-47b0-b7fd-a4615248fd03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:03.152946 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb22fb-79c0-47b0-b7fd-a4615248fd03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:03.155284 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb22fb-79c0-47b0-b7fd-a4615248fd03 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:03.179245 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:03.197564 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1017/2045] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:03 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:03.202274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2cdd9-e6f3-4d77-be7d-60b01b73193a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:03.217927 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2cdd9-e6f3-4d77-be7d-60b01b73193a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:03.296598 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b45050-26cf-4ecf-b1af-a2304b42d8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:03.297309 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b45050-26cf-4ecf-b1af-a2304b42d8eb admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:03.297585 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b45050-26cf-4ecf-b1af-a2304b42d8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:03.297861 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b45050-26cf-4ecf-b1af-a2304b42d8eb admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:03.298607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b45050-26cf-4ecf-b1af-a2304b42d8eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:03.300735 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b45050-26cf-4ecf-b1af-a2304b42d8eb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:03.326478 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1029/2046] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:03 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:03.331430 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb22fb-79c0-47b0-b7fd-a4615248fd03 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:03.342778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb22fb-79c0-47b0-b7fd-a4615248fd03 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:03.347824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c658d-6d3a-49ca-8b93-440554d48064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:03.348558 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c658d-6d3a-49ca-8b93-440554d48064 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:03.348766 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c658d-6d3a-49ca-8b93-440554d48064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:03.349039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c658d-6d3a-49ca-8b93-440554d48064 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:03.349701 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c658d-6d3a-49ca-8b93-440554d48064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:03.352005 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c658d-6d3a-49ca-8b93-440554d48064 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:03.365663 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1018/2047] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:03.370905 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b45050-26cf-4ecf-b1af-a2304b42d8eb admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:03.384349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b45050-26cf-4ecf-b1af-a2304b42d8eb admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:03.396940 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da8480-df6e-4c42-89c9-e4b840a367a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:03.397757 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da8480-df6e-4c42-89c9-e4b840a367a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:03.398110 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da8480-df6e-4c42-89c9-e4b840a367a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:03.398540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da8480-df6e-4c42-89c9-e4b840a367a5 admin admin] PATH_INFO: `/v3/projects/3999b7f83bf947998269872a07fe9e79/users/1d09255252d1454c8873a5f960c33068/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:03.417923 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da8480-df6e-4c42-89c9-e4b840a367a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3999b7f83bf947998269872a07fe9e79, user_id=1d09255252d1454c8873a5f960c33068, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:03.421025 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da8480-df6e-4c42-89c9-e4b840a367a5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:03.452241 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1030/2048] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:03 2022] PUT /identity/v3/projects/3999b7f83bf947998269872a07fe9e79/users/1d09255252d1454c8873a5f960c33068/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:03.456464 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c658d-6d3a-49ca-8b93-440554d48064 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:03.468893 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c658d-6d3a-49ca-8b93-440554d48064 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:03.535232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf2357-75cc-403a-b111-9f4ce0d35474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:03.535774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf2357-75cc-403a-b111-9f4ce0d35474 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:03.536059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf2357-75cc-403a-b111-9f4ce0d35474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:03.536383 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf2357-75cc-403a-b111-9f4ce0d35474 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:03.537070 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf2357-75cc-403a-b111-9f4ce0d35474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:03.538717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf2357-75cc-403a-b111-9f4ce0d35474 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:03.557168 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1019/2049] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:03.560002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da8480-df6e-4c42-89c9-e4b840a367a5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:03.571205 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da8480-df6e-4c42-89c9-e4b840a367a5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:03.575835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461a798-c3ec-43f0-b113-ca99d1b66eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:03.576533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461a798-c3ec-43f0-b113-ca99d1b66eeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:03.576817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461a798-c3ec-43f0-b113-ca99d1b66eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:03.577085 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461a798-c3ec-43f0-b113-ca99d1b66eeb admin admin] PATH_INFO: `/v3/projects/3999b7f83bf947998269872a07fe9e79/users/1d09255252d1454c8873a5f960c33068/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:03.578999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461a798-c3ec-43f0-b113-ca99d1b66eeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3999b7f83bf947998269872a07fe9e79, user_id=1d09255252d1454c8873a5f960c33068, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:03.581664 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461a798-c3ec-43f0-b113-ca99d1b66eeb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:03.594013 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1031/2050] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:03 2022] PUT /identity/v3/projects/3999b7f83bf947998269872a07fe9e79/users/1d09255252d1454c8873a5f960c33068/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:03.606729 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10840be2-1fd0-4545-8d99-469402911b74 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:03.606984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10840be2-1fd0-4545-8d99-469402911b74 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:03.607172 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10840be2-1fd0-4545-8d99-469402911b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:03.641430 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-10840be2-1fd0-4545-8d99-469402911b74 None None] MFA Rules not processed for user `1d09255252d1454c8873a5f960c33068`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:03.642819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-10840be2-1fd0-4545-8d99-469402911b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:03.755346 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1020/2051] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:03.767953 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461a798-c3ec-43f0-b113-ca99d1b66eeb admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:03.769745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9461a798-c3ec-43f0-b113-ca99d1b66eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:03.852267 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461a798-c3ec-43f0-b113-ca99d1b66eeb admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:03.856955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f045b8a-a62a-4099-adf6-3fd1db5ad41d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:03.857581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f045b8a-a62a-4099-adf6-3fd1db5ad41d service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:03.858159 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f045b8a-a62a-4099-adf6-3fd1db5ad41d service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:03.859059 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f045b8a-a62a-4099-adf6-3fd1db5ad41d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:03.870973 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f045b8a-a62a-4099-adf6-3fd1db5ad41d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:03.873031 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f045b8a-a62a-4099-adf6-3fd1db5ad41d service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:03.895754 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1032/2052] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:58:04.403954 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10840be2-1fd0-4545-8d99-469402911b74 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:04.490161 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10840be2-1fd0-4545-8d99-469402911b74 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:04.495970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4be790-8dd2-4849-b543-2e7b138d2ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:04.496920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4be790-8dd2-4849-b543-2e7b138d2ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:04.497245 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4be790-8dd2-4849-b543-2e7b138d2ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:04.497563 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4be790-8dd2-4849-b543-2e7b138d2ea1 admin admin] PATH_INFO: `/v3/users/709e0a7585674796ad2d61b560bc7350` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:04.498802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4be790-8dd2-4849-b543-2e7b138d2ea1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=709e0a7585674796ad2d61b560bc7350)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:04.500942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4be790-8dd2-4849-b543-2e7b138d2ea1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:04.507835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:04.563118 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5e4be790-8dd2-4849-b543-2e7b138d2ea1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '709e0a7585674796ad2d61b560bc7350'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:04.576407 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5e4be790-8dd2-4849-b543-2e7b138d2ea1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '709e0a7585674796ad2d61b560bc7350'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:04.591622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5e4be790-8dd2-4849-b543-2e7b138d2ea1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '709e0a7585674796ad2d61b560bc7350'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:04.598883 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5e4be790-8dd2-4849-b543-2e7b138d2ea1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '709e0a7585674796ad2d61b560bc7350'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:04.599673 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5e4be790-8dd2-4849-b543-2e7b138d2ea1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '709e0a7585674796ad2d61b560bc7350'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:04.606586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1021/2053] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:04 2022] DELETE /identity/v3/users/709e0a7585674796ad2d61b560bc7350 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:04.611188 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a3612-5e1e-4562-a794-42f8d7e48035 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:04.611472 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a3612-5e1e-4562-a794-42f8d7e48035 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:04.611746 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a3612-5e1e-4562-a794-42f8d7e48035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:04.638783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5c0a3612-5e1e-4562-a794-42f8d7e48035 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:04.640214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5c0a3612-5e1e-4562-a794-42f8d7e48035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:04.650025 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:04.716063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1033/2054] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:04.728280 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4be790-8dd2-4849-b543-2e7b138d2ea1 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:04.729746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5e4be790-8dd2-4849-b543-2e7b138d2ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:04.793727 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4be790-8dd2-4849-b543-2e7b138d2ea1 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:04.797705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98248292-60f6-4f30-8a3d-64e0436ca96d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:04.798396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98248292-60f6-4f30-8a3d-64e0436ca96d service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:04.798619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98248292-60f6-4f30-8a3d-64e0436ca96d service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:04.798886 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98248292-60f6-4f30-8a3d-64e0436ca96d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:04.811524 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98248292-60f6-4f30-8a3d-64e0436ca96d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:04.813015 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98248292-60f6-4f30-8a3d-64e0436ca96d service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:04.823717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1022/2055] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:05.028604 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af4300-4672-4250-917d-be72f10c33c3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:05.029173 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af4300-4672-4250-917d-be72f10c33c3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:05.029496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af4300-4672-4250-917d-be72f10c33c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:05.049743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-56af4300-4672-4250-917d-be72f10c33c3 None None] MFA Rules not processed for user `3095e7f94a58497c812890fa86e08eec`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:05.050939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-56af4300-4672-4250-917d-be72f10c33c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:05.131347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98248292-60f6-4f30-8a3d-64e0436ca96d service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:05.133146 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-98248292-60f6-4f30-8a3d-64e0436ca96d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:05.137420 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1034/2056] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:05.149190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98248292-60f6-4f30-8a3d-64e0436ca96d service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:05.153694 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23653a-042e-4cf8-8fbe-44ba334534f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:05.154462 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23653a-042e-4cf8-8fbe-44ba334534f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:05.155140 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23653a-042e-4cf8-8fbe-44ba334534f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:05.155505 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23653a-042e-4cf8-8fbe-44ba334534f1 admin admin] PATH_INFO: `/v3/projects/15af91597a02408a9516533b5153b214` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:05.157173 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23653a-042e-4cf8-8fbe-44ba334534f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15af91597a02408a9516533b5153b214)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:05.159424 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23653a-042e-4cf8-8fbe-44ba334534f1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:05.233547 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-dc23653a-042e-4cf8-8fbe-44ba334534f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15af91597a02408a9516533b5153b214', 'request_id': 'req-dc23653a-042e-4cf8-8fbe-44ba334534f1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:05.242089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-dc23653a-042e-4cf8-8fbe-44ba334534f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15af91597a02408a9516533b5153b214', 'request_id': 'req-dc23653a-042e-4cf8-8fbe-44ba334534f1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:05.248439 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-dc23653a-042e-4cf8-8fbe-44ba334534f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15af91597a02408a9516533b5153b214', 'request_id': 'req-dc23653a-042e-4cf8-8fbe-44ba334534f1'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:05.260392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-dc23653a-042e-4cf8-8fbe-44ba334534f1 admin admin] The token cache is being invalidate because project 15af91597a02408a9516533b5153b214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:05.260682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-dc23653a-042e-4cf8-8fbe-44ba334534f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:05.261365 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-dc23653a-042e-4cf8-8fbe-44ba334534f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:05.262895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1023/2057] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:05 2022] DELETE /identity/v3/projects/15af91597a02408a9516533b5153b214 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:05.266918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af4300-4672-4250-917d-be72f10c33c3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:05.278111 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af4300-4672-4250-917d-be72f10c33c3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:05.341338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef87cc5-6e7a-4b72-9787-a2cba9a6d444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:05.341990 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef87cc5-6e7a-4b72-9787-a2cba9a6d444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:05.342317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef87cc5-6e7a-4b72-9787-a2cba9a6d444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:05.342598 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef87cc5-6e7a-4b72-9787-a2cba9a6d444 admin admin] PATH_INFO: `/v3/users/1d09255252d1454c8873a5f960c33068` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:05.343935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef87cc5-6e7a-4b72-9787-a2cba9a6d444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d09255252d1454c8873a5f960c33068)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:05.346108 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef87cc5-6e7a-4b72-9787-a2cba9a6d444 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:05.403820 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0ef87cc5-6e7a-4b72-9787-a2cba9a6d444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d09255252d1454c8873a5f960c33068'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:05.407070 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0ef87cc5-6e7a-4b72-9787-a2cba9a6d444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d09255252d1454c8873a5f960c33068'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:05.424856 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0ef87cc5-6e7a-4b72-9787-a2cba9a6d444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d09255252d1454c8873a5f960c33068'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:05.431959 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:05.434829 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0ef87cc5-6e7a-4b72-9787-a2cba9a6d444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d09255252d1454c8873a5f960c33068'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:05.451614 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0ef87cc5-6e7a-4b72-9787-a2cba9a6d444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d09255252d1454c8873a5f960c33068'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:05.462373 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1035/2058] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:05 2022] DELETE /identity/v3/users/1d09255252d1454c8873a5f960c33068 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:05.871997 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23653a-042e-4cf8-8fbe-44ba334534f1 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:05.873885 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dc23653a-042e-4cf8-8fbe-44ba334534f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:05.888668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23653a-042e-4cf8-8fbe-44ba334534f1 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:05.899907 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:05.992288 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891bd7d-8a6d-44d9-bd0b-16c27ca85035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:05.992288 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891bd7d-8a6d-44d9-bd0b-16c27ca85035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:05.992288 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891bd7d-8a6d-44d9-bd0b-16c27ca85035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:05.992288 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891bd7d-8a6d-44d9-bd0b-16c27ca85035 admin admin] PATH_INFO: `/v3/projects/3999b7f83bf947998269872a07fe9e79` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:05.993282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891bd7d-8a6d-44d9-bd0b-16c27ca85035 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3999b7f83bf947998269872a07fe9e79)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:05.995202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891bd7d-8a6d-44d9-bd0b-16c27ca85035 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:06.048865 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5891bd7d-8a6d-44d9-bd0b-16c27ca85035 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3999b7f83bf947998269872a07fe9e79', 'request_id': 'req-5891bd7d-8a6d-44d9-bd0b-16c27ca85035'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:06.057447 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5891bd7d-8a6d-44d9-bd0b-16c27ca85035 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3999b7f83bf947998269872a07fe9e79', 'request_id': 'req-5891bd7d-8a6d-44d9-bd0b-16c27ca85035'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:06.063463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5891bd7d-8a6d-44d9-bd0b-16c27ca85035 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3999b7f83bf947998269872a07fe9e79', 'request_id': 'req-5891bd7d-8a6d-44d9-bd0b-16c27ca85035'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:06.079621 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5891bd7d-8a6d-44d9-bd0b-16c27ca85035 admin admin] The token cache is being invalidate because project 3999b7f83bf947998269872a07fe9e79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:06.080058 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5891bd7d-8a6d-44d9-bd0b-16c27ca85035 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:06.080929 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5891bd7d-8a6d-44d9-bd0b-16c27ca85035 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:06.082928 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1024/2059] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:05 2022] DELETE /identity/v3/projects/3999b7f83bf947998269872a07fe9e79 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:06.088494 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da3f96-2532-467b-b5a1-32761043aaed None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:06.089005 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da3f96-2532-467b-b5a1-32761043aaed None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:06.089469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da3f96-2532-467b-b5a1-32761043aaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:06.109997 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-99da3f96-2532-467b-b5a1-32761043aaed None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:06.111283 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-99da3f96-2532-467b-b5a1-32761043aaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:06.180887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1036/2060] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:06.190161 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35887ce0-6022-43c9-b234-2fbb79778c68 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:06.190470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35887ce0-6022-43c9-b234-2fbb79778c68 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:06.190766 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35887ce0-6022-43c9-b234-2fbb79778c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:06.213026 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-35887ce0-6022-43c9-b234-2fbb79778c68 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:06.214254 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-35887ce0-6022-43c9-b234-2fbb79778c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:06.223076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1025/2061] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:06.228164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da3f96-2532-467b-b5a1-32761043aaed None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:06.246967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da3f96-2532-467b-b5a1-32761043aaed None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:06.253923 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81784db8-4920-44db-9b9f-e22088f28f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:06.254702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81784db8-4920-44db-9b9f-e22088f28f81 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:06.254990 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81784db8-4920-44db-9b9f-e22088f28f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:06.255266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81784db8-4920-44db-9b9f-e22088f28f81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:06.257484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81784db8-4920-44db-9b9f-e22088f28f81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:06.257484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81784db8-4920-44db-9b9f-e22088f28f81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:06.259198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81784db8-4920-44db-9b9f-e22088f28f81 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:06.273178 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1037/2062] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:58:06.277283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35887ce0-6022-43c9-b234-2fbb79778c68 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:06.288676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35887ce0-6022-43c9-b234-2fbb79778c68 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:06.296549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ed88f-96b4-4564-a5b3-5257727b22f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:06.297138 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ed88f-96b4-4564-a5b3-5257727b22f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:06.297414 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ed88f-96b4-4564-a5b3-5257727b22f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:06.297748 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ed88f-96b4-4564-a5b3-5257727b22f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:06.298387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ed88f-96b4-4564-a5b3-5257727b22f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:06.300716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ed88f-96b4-4564-a5b3-5257727b22f5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:06.342564 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1026/2063] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:06 2022] POST /identity/v3/projects => generated 357 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:06.346804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81784db8-4920-44db-9b9f-e22088f28f81 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:06.359679 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81784db8-4920-44db-9b9f-e22088f28f81 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:06.424464 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21546862-0bde-4cff-abaa-5ed70b80b5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:06.425066 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21546862-0bde-4cff-abaa-5ed70b80b5e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:06.425416 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21546862-0bde-4cff-abaa-5ed70b80b5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:06.425759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21546862-0bde-4cff-abaa-5ed70b80b5e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:06.426558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21546862-0bde-4cff-abaa-5ed70b80b5e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:06.429346 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21546862-0bde-4cff-abaa-5ed70b80b5e4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:06.453981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1038/2064] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:06 2022] POST /identity/v3/users => generated 279 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:06.459680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ed88f-96b4-4564-a5b3-5257727b22f5 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:06.472657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ed88f-96b4-4564-a5b3-5257727b22f5 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:06.479211 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd3905-ddd5-44eb-b12c-5d31a539718a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:06.479923 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd3905-ddd5-44eb-b12c-5d31a539718a admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:06.480630 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd3905-ddd5-44eb-b12c-5d31a539718a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:06.480630 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd3905-ddd5-44eb-b12c-5d31a539718a admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:06.481466 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd3905-ddd5-44eb-b12c-5d31a539718a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:06.483214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd3905-ddd5-44eb-b12c-5d31a539718a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:06.495191 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1027/2065] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:06.499362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21546862-0bde-4cff-abaa-5ed70b80b5e4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:06.511309 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21546862-0bde-4cff-abaa-5ed70b80b5e4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:06.516678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db278b-f586-46f0-bb10-148c0e401c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:06.517330 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db278b-f586-46f0-bb10-148c0e401c1e admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:06.517614 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db278b-f586-46f0-bb10-148c0e401c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:06.517888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db278b-f586-46f0-bb10-148c0e401c1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:06.518555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db278b-f586-46f0-bb10-148c0e401c1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:06.520213 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db278b-f586-46f0-bb10-148c0e401c1e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:06.533025 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1039/2066] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:06.537314 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd3905-ddd5-44eb-b12c-5d31a539718a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:06.547982 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd3905-ddd5-44eb-b12c-5d31a539718a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:06.554516 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e4013-b8e7-4c05-b4aa-deeeb86ea79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:06.555682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e4013-b8e7-4c05-b4aa-deeeb86ea79e admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:06.555945 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e4013-b8e7-4c05-b4aa-deeeb86ea79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:06.556308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e4013-b8e7-4c05-b4aa-deeeb86ea79e admin admin] PATH_INFO: `/v3/projects/509c1249240e4bf189faa6b77528b012/users/fe0456b21ade421f89daf89f27394df9/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:06.575526 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e4013-b8e7-4c05-b4aa-deeeb86ea79e admin admin] RBAC: Authorizing `identity:create_grant(project_id=509c1249240e4bf189faa6b77528b012, user_id=fe0456b21ade421f89daf89f27394df9, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:06.578536 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e4013-b8e7-4c05-b4aa-deeeb86ea79e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:06.593008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1028/2067] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:06 2022] PUT /identity/v3/projects/509c1249240e4bf189faa6b77528b012/users/fe0456b21ade421f89daf89f27394df9/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:06.606159 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd9038-0e65-4158-9919-016222a3645c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:06.606714 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd9038-0e65-4158-9919-016222a3645c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:06.607178 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd9038-0e65-4158-9919-016222a3645c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:06.648453 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-abfd9038-0e65-4158-9919-016222a3645c None None] MFA Rules not processed for user `fe0456b21ade421f89daf89f27394df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:06.649983 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-abfd9038-0e65-4158-9919-016222a3645c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:06.688451 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:06.727467 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1040/2068] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:06 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:06.732068 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e4013-b8e7-4c05-b4aa-deeeb86ea79e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:06.776413 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:06.805344 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e4013-b8e7-4c05-b4aa-deeeb86ea79e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:06.810424 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09583b8e-18a8-44ac-8c3e-2d3aa939d59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:06.810810 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09583b8e-18a8-44ac-8c3e-2d3aa939d59c admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:06.811089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09583b8e-18a8-44ac-8c3e-2d3aa939d59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:06.811423 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09583b8e-18a8-44ac-8c3e-2d3aa939d59c admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:06.812069 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09583b8e-18a8-44ac-8c3e-2d3aa939d59c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:06.814001 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09583b8e-18a8-44ac-8c3e-2d3aa939d59c admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:06.853196 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1029/2069] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:06 2022] POST /identity/v3/projects => generated 355 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:06.859129 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd9038-0e65-4158-9919-016222a3645c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:06.872061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd9038-0e65-4158-9919-016222a3645c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:06.955499 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f0bbe-062d-4e40-88eb-1137719de924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:06.956311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f0bbe-062d-4e40-88eb-1137719de924 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:06.956458 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f0bbe-062d-4e40-88eb-1137719de924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:06.956768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f0bbe-062d-4e40-88eb-1137719de924 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:06.957355 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f0bbe-062d-4e40-88eb-1137719de924 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:06.959324 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f0bbe-062d-4e40-88eb-1137719de924 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:06.996149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1041/2070] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:06 2022] POST /identity/v3/users => generated 284 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:06.999188 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09583b8e-18a8-44ac-8c3e-2d3aa939d59c admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:07.014127 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09583b8e-18a8-44ac-8c3e-2d3aa939d59c admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:07.022141 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36245d-b180-400a-8eda-db6c6807dfc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:07.023319 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36245d-b180-400a-8eda-db6c6807dfc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:07.023700 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36245d-b180-400a-8eda-db6c6807dfc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:07.024500 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36245d-b180-400a-8eda-db6c6807dfc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:07.025439 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d36245d-b180-400a-8eda-db6c6807dfc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:07.030670 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d36245d-b180-400a-8eda-db6c6807dfc6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:07.046198 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1030/2071] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:58:07.052019 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f0bbe-062d-4e40-88eb-1137719de924 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:07.065219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f0bbe-062d-4e40-88eb-1137719de924 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:07.078955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03096756-4f5a-4452-abb6-3984a3b1888b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:07.079693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03096756-4f5a-4452-abb6-3984a3b1888b admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:07.079971 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03096756-4f5a-4452-abb6-3984a3b1888b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:07.080292 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03096756-4f5a-4452-abb6-3984a3b1888b admin admin] PATH_INFO: `/v3/projects/39eb7aae489144ac813079b8504f23b9/users/eeb3471e3be34c9ebd9705e7eef2ef91/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:07.104529 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03096756-4f5a-4452-abb6-3984a3b1888b admin admin] RBAC: Authorizing `identity:create_grant(project_id=39eb7aae489144ac813079b8504f23b9, user_id=eeb3471e3be34c9ebd9705e7eef2ef91, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:07.107987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03096756-4f5a-4452-abb6-3984a3b1888b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:07.131540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1042/2072] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:07 2022] PUT /identity/v3/projects/39eb7aae489144ac813079b8504f23b9/users/eeb3471e3be34c9ebd9705e7eef2ef91/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:07.135319 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36245d-b180-400a-8eda-db6c6807dfc6 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:07.151667 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36245d-b180-400a-8eda-db6c6807dfc6 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:07.255785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fc3c2-b2ec-4d06-bed7-4e9006ba502b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:07.256621 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fc3c2-b2ec-4d06-bed7-4e9006ba502b admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:07.257653 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fc3c2-b2ec-4d06-bed7-4e9006ba502b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:07.257827 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fc3c2-b2ec-4d06-bed7-4e9006ba502b admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:07.258603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31fc3c2-b2ec-4d06-bed7-4e9006ba502b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:07.260346 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31fc3c2-b2ec-4d06-bed7-4e9006ba502b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:07.272584 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1031/2073] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:07.277083 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03096756-4f5a-4452-abb6-3984a3b1888b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:07.300962 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03096756-4f5a-4452-abb6-3984a3b1888b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:07.309868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64b01b-100c-4d5f-948e-3a2f19613075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:07.311099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64b01b-100c-4d5f-948e-3a2f19613075 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:07.311542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64b01b-100c-4d5f-948e-3a2f19613075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:07.311890 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64b01b-100c-4d5f-948e-3a2f19613075 admin admin] PATH_INFO: `/v3/projects/39eb7aae489144ac813079b8504f23b9/users/eeb3471e3be34c9ebd9705e7eef2ef91/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:07.314982 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd64b01b-100c-4d5f-948e-3a2f19613075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39eb7aae489144ac813079b8504f23b9, user_id=eeb3471e3be34c9ebd9705e7eef2ef91, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:07.317933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd64b01b-100c-4d5f-948e-3a2f19613075 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:07.332443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1043/2074] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:07 2022] PUT /identity/v3/projects/39eb7aae489144ac813079b8504f23b9/users/eeb3471e3be34c9ebd9705e7eef2ef91/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:07.343524 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7671a5-ceb8-4e88-8b8d-4d61ee3d5fa1 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:07.343816 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7671a5-ceb8-4e88-8b8d-4d61ee3d5fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:07.344114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7671a5-ceb8-4e88-8b8d-4d61ee3d5fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:07.380676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-ed7671a5-ceb8-4e88-8b8d-4d61ee3d5fa1 None None] MFA Rules not processed for user `eeb3471e3be34c9ebd9705e7eef2ef91`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:07.382087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ed7671a5-ceb8-4e88-8b8d-4d61ee3d5fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:07.470448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1032/2075] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:07 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:07.485386 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64b01b-100c-4d5f-948e-3a2f19613075 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:07.487341 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fd64b01b-100c-4d5f-948e-3a2f19613075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:07.564643 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64b01b-100c-4d5f-948e-3a2f19613075 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:07.570160 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5cc9f-3ea5-45d1-b8f9-e46e38ecadd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:07.571182 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5cc9f-3ea5-45d1-b8f9-e46e38ecadd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:07.571733 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5cc9f-3ea5-45d1-b8f9-e46e38ecadd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:07.572383 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5cc9f-3ea5-45d1-b8f9-e46e38ecadd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:07.583817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5cc9f-3ea5-45d1-b8f9-e46e38ecadd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:07.585708 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5cc9f-3ea5-45d1-b8f9-e46e38ecadd8 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:07.596732 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1044/2076] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:07 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:07.686518 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7671a5-ceb8-4e88-8b8d-4d61ee3d5fa1 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:07.697668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7671a5-ceb8-4e88-8b8d-4d61ee3d5fa1 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:07.701608 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c085a3d-6cf2-4ef4-9ef8-43d71af45569 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:07.702219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c085a3d-6cf2-4ef4-9ef8-43d71af45569 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:07.702495 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c085a3d-6cf2-4ef4-9ef8-43d71af45569 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:07.702762 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c085a3d-6cf2-4ef4-9ef8-43d71af45569 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:07.740061 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:07.772720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c085a3d-6cf2-4ef4-9ef8-43d71af45569 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:07.774274 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c085a3d-6cf2-4ef4-9ef8-43d71af45569 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:07.790680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1033/2077] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:07 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:17.591974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:17.781476 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=14 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:24.333379 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5cc9f-3ea5-45d1-b8f9-e46e38ecadd8 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:24.405977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5cc9f-3ea5-45d1-b8f9-e46e38ecadd8 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:24.410542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c2378-5653-4db6-96cb-50079115cc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:24.411179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c2378-5653-4db6-96cb-50079115cc2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:24.411460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c2378-5653-4db6-96cb-50079115cc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:24.411818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c2378-5653-4db6-96cb-50079115cc2a admin admin] PATH_INFO: `/v3/users/fe0456b21ade421f89daf89f27394df9` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:24.413015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c2378-5653-4db6-96cb-50079115cc2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe0456b21ade421f89daf89f27394df9)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:24.415313 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c2378-5653-4db6-96cb-50079115cc2a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:24.480169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-822c2378-5653-4db6-96cb-50079115cc2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe0456b21ade421f89daf89f27394df9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:24.480896 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-822c2378-5653-4db6-96cb-50079115cc2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe0456b21ade421f89daf89f27394df9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:24.490896 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-822c2378-5653-4db6-96cb-50079115cc2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe0456b21ade421f89daf89f27394df9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:24.500502 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-822c2378-5653-4db6-96cb-50079115cc2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe0456b21ade421f89daf89f27394df9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:24.515730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-822c2378-5653-4db6-96cb-50079115cc2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe0456b21ade421f89daf89f27394df9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:24.522463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1045/2078] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:24 2022] DELETE /identity/v3/users/fe0456b21ade421f89daf89f27394df9 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:24.528007 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd21d0-5a4d-4516-83bb-9154d8dd8a9e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:24.528581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd21d0-5a4d-4516-83bb-9154d8dd8a9e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:24.528581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd21d0-5a4d-4516-83bb-9154d8dd8a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:24.549044 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-05cd21d0-5a4d-4516-83bb-9154d8dd8a9e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:24.550283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-05cd21d0-5a4d-4516-83bb-9154d8dd8a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:24.625128 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1034/2079] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:24.635567 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c2378-5653-4db6-96cb-50079115cc2a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:24.637163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-822c2378-5653-4db6-96cb-50079115cc2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:24.702015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c2378-5653-4db6-96cb-50079115cc2a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:24.705934 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61a75a-14c4-470b-a4d5-0a76ce5a6e04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:24.706543 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61a75a-14c4-470b-a4d5-0a76ce5a6e04 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:24.706819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61a75a-14c4-470b-a4d5-0a76ce5a6e04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:24.707095 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61a75a-14c4-470b-a4d5-0a76ce5a6e04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:24.724800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61a75a-14c4-470b-a4d5-0a76ce5a6e04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:24.726255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61a75a-14c4-470b-a4d5-0a76ce5a6e04 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:24.739220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1046/2080] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:25.131330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd21d0-5a4d-4516-83bb-9154d8dd8a9e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:25.133617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-05cd21d0-5a4d-4516-83bb-9154d8dd8a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:25.147863 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd21d0-5a4d-4516-83bb-9154d8dd8a9e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:25.156898 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ddb8b-0ea1-411c-a8ec-17aacfb4e55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:25.157673 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ddb8b-0ea1-411c-a8ec-17aacfb4e55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:25.158015 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ddb8b-0ea1-411c-a8ec-17aacfb4e55f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:25.158233 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ddb8b-0ea1-411c-a8ec-17aacfb4e55f admin admin] PATH_INFO: `/v3/projects/509c1249240e4bf189faa6b77528b012` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:25.160128 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ddb8b-0ea1-411c-a8ec-17aacfb4e55f admin admin] RBAC: Authorizing `identity:delete_project(project_id=509c1249240e4bf189faa6b77528b012)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:25.162512 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ddb8b-0ea1-411c-a8ec-17aacfb4e55f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:25.221131 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-670ddb8b-0ea1-411c-a8ec-17aacfb4e55f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '509c1249240e4bf189faa6b77528b012', 'request_id': 'req-670ddb8b-0ea1-411c-a8ec-17aacfb4e55f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.229593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-670ddb8b-0ea1-411c-a8ec-17aacfb4e55f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '509c1249240e4bf189faa6b77528b012', 'request_id': 'req-670ddb8b-0ea1-411c-a8ec-17aacfb4e55f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.237011 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-670ddb8b-0ea1-411c-a8ec-17aacfb4e55f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '509c1249240e4bf189faa6b77528b012', 'request_id': 'req-670ddb8b-0ea1-411c-a8ec-17aacfb4e55f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.250550 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-670ddb8b-0ea1-411c-a8ec-17aacfb4e55f admin admin] The token cache is being invalidate because project 509c1249240e4bf189faa6b77528b012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:25.250929 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-670ddb8b-0ea1-411c-a8ec-17aacfb4e55f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.251817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-670ddb8b-0ea1-411c-a8ec-17aacfb4e55f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.253818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1035/2081] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:25 2022] DELETE /identity/v3/projects/509c1249240e4bf189faa6b77528b012 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:25.265813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61a75a-14c4-470b-a4d5-0a76ce5a6e04 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:25.277343 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61a75a-14c4-470b-a4d5-0a76ce5a6e04 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:25.304308 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:25.351698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9ea14-8b95-4942-97d3-3ef9c7d0c336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:25.352621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9ea14-8b95-4942-97d3-3ef9c7d0c336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:25.352906 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9ea14-8b95-4942-97d3-3ef9c7d0c336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:25.353186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9ea14-8b95-4942-97d3-3ef9c7d0c336 admin admin] PATH_INFO: `/v3/users/eeb3471e3be34c9ebd9705e7eef2ef91` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:25.354463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc9ea14-8b95-4942-97d3-3ef9c7d0c336 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeb3471e3be34c9ebd9705e7eef2ef91)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:25.356404 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc9ea14-8b95-4942-97d3-3ef9c7d0c336 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:25.410776 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2fc9ea14-8b95-4942-97d3-3ef9c7d0c336 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeb3471e3be34c9ebd9705e7eef2ef91'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.411730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2fc9ea14-8b95-4942-97d3-3ef9c7d0c336 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeb3471e3be34c9ebd9705e7eef2ef91'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.430180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2fc9ea14-8b95-4942-97d3-3ef9c7d0c336 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeb3471e3be34c9ebd9705e7eef2ef91'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.442934 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2fc9ea14-8b95-4942-97d3-3ef9c7d0c336 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeb3471e3be34c9ebd9705e7eef2ef91'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.460863 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2fc9ea14-8b95-4942-97d3-3ef9c7d0c336 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeb3471e3be34c9ebd9705e7eef2ef91'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.469684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1047/2082] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:25 2022] DELETE /identity/v3/users/eeb3471e3be34c9ebd9705e7eef2ef91 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:25.755799 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ddb8b-0ea1-411c-a8ec-17aacfb4e55f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:25.757656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-670ddb8b-0ea1-411c-a8ec-17aacfb4e55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:25.780433 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ddb8b-0ea1-411c-a8ec-17aacfb4e55f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:25.846426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b96c5c-a9b0-4f1a-939b-17ec634b7bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:25.847066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b96c5c-a9b0-4f1a-939b-17ec634b7bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:25.847339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b96c5c-a9b0-4f1a-939b-17ec634b7bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:25.847614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b96c5c-a9b0-4f1a-939b-17ec634b7bb8 admin admin] PATH_INFO: `/v3/projects/39eb7aae489144ac813079b8504f23b9` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:25.848936 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b96c5c-a9b0-4f1a-939b-17ec634b7bb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39eb7aae489144ac813079b8504f23b9)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:25.850869 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b96c5c-a9b0-4f1a-939b-17ec634b7bb8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:25.890999 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:25.909939 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-29b96c5c-a9b0-4f1a-939b-17ec634b7bb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39eb7aae489144ac813079b8504f23b9', 'request_id': 'req-29b96c5c-a9b0-4f1a-939b-17ec634b7bb8'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.919377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-29b96c5c-a9b0-4f1a-939b-17ec634b7bb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39eb7aae489144ac813079b8504f23b9', 'request_id': 'req-29b96c5c-a9b0-4f1a-939b-17ec634b7bb8'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.928976 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-29b96c5c-a9b0-4f1a-939b-17ec634b7bb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39eb7aae489144ac813079b8504f23b9', 'request_id': 'req-29b96c5c-a9b0-4f1a-939b-17ec634b7bb8'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.943805 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-29b96c5c-a9b0-4f1a-939b-17ec634b7bb8 admin admin] The token cache is being invalidate because project 39eb7aae489144ac813079b8504f23b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:25.944296 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-29b96c5c-a9b0-4f1a-939b-17ec634b7bb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.945150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-29b96c5c-a9b0-4f1a-939b-17ec634b7bb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:25.946821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1036/2083] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:25 2022] DELETE /identity/v3/projects/39eb7aae489144ac813079b8504f23b9 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:25.951953 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430874cd-e8eb-41c4-a5a5-636d8c822530 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:25.952275 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430874cd-e8eb-41c4-a5a5-636d8c822530 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:25.952572 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430874cd-e8eb-41c4-a5a5-636d8c822530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:25.972704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-430874cd-e8eb-41c4-a5a5-636d8c822530 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:25.973907 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-430874cd-e8eb-41c4-a5a5-636d8c822530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:26.037056 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1048/2084] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:26.045325 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905a525-4bec-4c3c-b7dc-cc3363360f25 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:26.045771 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905a525-4bec-4c3c-b7dc-cc3363360f25 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:26.046053 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905a525-4bec-4c3c-b7dc-cc3363360f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:26.067567 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b905a525-4bec-4c3c-b7dc-cc3363360f25 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:26.068826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b905a525-4bec-4c3c-b7dc-cc3363360f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:26.079202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1037/2085] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:26.083574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430874cd-e8eb-41c4-a5a5-636d8c822530 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:26.093776 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430874cd-e8eb-41c4-a5a5-636d8c822530 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:26.098404 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5971ac-061f-4230-b40a-40aa98028e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:26.099040 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5971ac-061f-4230-b40a-40aa98028e1b admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:26.099315 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5971ac-061f-4230-b40a-40aa98028e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:26.099584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5971ac-061f-4230-b40a-40aa98028e1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:26.100319 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5971ac-061f-4230-b40a-40aa98028e1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:26.100622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5971ac-061f-4230-b40a-40aa98028e1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:26.102359 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5971ac-061f-4230-b40a-40aa98028e1b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:26.115106 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1049/2086] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:58:26.120369 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905a525-4bec-4c3c-b7dc-cc3363360f25 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:26.130312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905a525-4bec-4c3c-b7dc-cc3363360f25 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:26.136831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b82625-6a7b-4312-b217-d8aa70ea86da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:26.137444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b82625-6a7b-4312-b217-d8aa70ea86da admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:26.137720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b82625-6a7b-4312-b217-d8aa70ea86da admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:26.137988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b82625-6a7b-4312-b217-d8aa70ea86da admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:26.138620 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b82625-6a7b-4312-b217-d8aa70ea86da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:26.140582 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b82625-6a7b-4312-b217-d8aa70ea86da admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:26.172413 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1038/2087] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:26 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:26.173535 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5971ac-061f-4230-b40a-40aa98028e1b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:26.184847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5971ac-061f-4230-b40a-40aa98028e1b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:26.216349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:26.251035 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8e500-2a67-431f-a3bb-5ec6c3669a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:26.251651 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8e500-2a67-431f-a3bb-5ec6c3669a9a admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:26.251934 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8e500-2a67-431f-a3bb-5ec6c3669a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:26.252250 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8e500-2a67-431f-a3bb-5ec6c3669a9a admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:26.252913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8e500-2a67-431f-a3bb-5ec6c3669a9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:26.254836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8e500-2a67-431f-a3bb-5ec6c3669a9a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:26.278859 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1050/2088] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:26 2022] POST /identity/v3/users => generated 282 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:26.283009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b82625-6a7b-4312-b217-d8aa70ea86da admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:26.294128 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b82625-6a7b-4312-b217-d8aa70ea86da admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:26.298745 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb90fcc-e5fc-46c0-9b54-e23cd89b436b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:26.299351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb90fcc-e5fc-46c0-9b54-e23cd89b436b admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:26.299630 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb90fcc-e5fc-46c0-9b54-e23cd89b436b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:26.299934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb90fcc-e5fc-46c0-9b54-e23cd89b436b admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:26.300622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb90fcc-e5fc-46c0-9b54-e23cd89b436b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:26.302211 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb90fcc-e5fc-46c0-9b54-e23cd89b436b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:26.313369 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1039/2089] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:26.318531 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8e500-2a67-431f-a3bb-5ec6c3669a9a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:26.329470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8e500-2a67-431f-a3bb-5ec6c3669a9a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:26.334428 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e50de-d5a6-4ee5-a5a7-2c132c6a1b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:26.335028 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e50de-d5a6-4ee5-a5a7-2c132c6a1b98 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:26.335299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e50de-d5a6-4ee5-a5a7-2c132c6a1b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:26.335571 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e50de-d5a6-4ee5-a5a7-2c132c6a1b98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:26.336836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e50de-d5a6-4ee5-a5a7-2c132c6a1b98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:26.338344 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e50de-d5a6-4ee5-a5a7-2c132c6a1b98 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:26.349134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1051/2090] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:26.354037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb90fcc-e5fc-46c0-9b54-e23cd89b436b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:26.364057 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb90fcc-e5fc-46c0-9b54-e23cd89b436b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:26.369446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904e239-768f-4710-8323-3c50847dc713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:26.370098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904e239-768f-4710-8323-3c50847dc713 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:26.370377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904e239-768f-4710-8323-3c50847dc713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:26.370679 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904e239-768f-4710-8323-3c50847dc713 admin admin] PATH_INFO: `/v3/projects/993f03f0e9344ee7b44767018ba3fab1/users/b913bfe9ba934806a4f17551d5dbbfea/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:26.386034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904e239-768f-4710-8323-3c50847dc713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=993f03f0e9344ee7b44767018ba3fab1, user_id=b913bfe9ba934806a4f17551d5dbbfea, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:26.388765 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904e239-768f-4710-8323-3c50847dc713 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:26.401922 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1040/2091] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:26 2022] PUT /identity/v3/projects/993f03f0e9344ee7b44767018ba3fab1/users/b913bfe9ba934806a4f17551d5dbbfea/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:26.411904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e6df8-c80d-44b3-a3ef-b7ed6e948cb9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:26.412423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e6df8-c80d-44b3-a3ef-b7ed6e948cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:26.412518 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e6df8-c80d-44b3-a3ef-b7ed6e948cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:26.444472 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-723e6df8-c80d-44b3-a3ef-b7ed6e948cb9 None None] MFA Rules not processed for user `b913bfe9ba934806a4f17551d5dbbfea`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:26.446213 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-723e6df8-c80d-44b3-a3ef-b7ed6e948cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:26.518788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1052/2092] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:26.523116 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904e239-768f-4710-8323-3c50847dc713 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:26.589591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904e239-768f-4710-8323-3c50847dc713 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:26.594570 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b3962-921b-43a8-a173-895953d654d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:26.595294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b3962-921b-43a8-a173-895953d654d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:26.595645 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b3962-921b-43a8-a173-895953d654d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:26.595990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b3962-921b-43a8-a173-895953d654d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:26.596667 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b3962-921b-43a8-a173-895953d654d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:26.598592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b3962-921b-43a8-a173-895953d654d3 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:26.631699 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1041/2093] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:26 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:26.636405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e6df8-c80d-44b3-a3ef-b7ed6e948cb9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:26.649255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e6df8-c80d-44b3-a3ef-b7ed6e948cb9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:26.712386 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0851d4-ae83-443b-91a2-ae1b755517ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:26.712996 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0851d4-ae83-443b-91a2-ae1b755517ae admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:26.713270 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0851d4-ae83-443b-91a2-ae1b755517ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:26.713534 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0851d4-ae83-443b-91a2-ae1b755517ae admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:26.714162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0851d4-ae83-443b-91a2-ae1b755517ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:26.716070 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0851d4-ae83-443b-91a2-ae1b755517ae admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:26.739943 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1053/2094] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:26 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:26.741155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b3962-921b-43a8-a173-895953d654d3 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:26.752161 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b3962-921b-43a8-a173-895953d654d3 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:26.757240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a0441-0fe9-4999-92f3-1be725a731ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:26.758389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a0441-0fe9-4999-92f3-1be725a731ae admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:26.758673 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a0441-0fe9-4999-92f3-1be725a731ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:26.758957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a0441-0fe9-4999-92f3-1be725a731ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:26.759678 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a0441-0fe9-4999-92f3-1be725a731ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:26.761311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a0441-0fe9-4999-92f3-1be725a731ae admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:26.771937 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1042/2095] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:26.777031 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0851d4-ae83-443b-91a2-ae1b755517ae admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:26.787083 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0851d4-ae83-443b-91a2-ae1b755517ae admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:26.791877 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d175f18-28db-4bd5-9e2e-8078fbb76010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:26.792569 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d175f18-28db-4bd5-9e2e-8078fbb76010 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:26.792852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d175f18-28db-4bd5-9e2e-8078fbb76010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:26.793125 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d175f18-28db-4bd5-9e2e-8078fbb76010 admin admin] PATH_INFO: `/v3/projects/3487d13d285441b6888deb400ca9122c/users/a1e4c30e975a4326a208053a48fc04fb/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:26.808832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d175f18-28db-4bd5-9e2e-8078fbb76010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3487d13d285441b6888deb400ca9122c, user_id=a1e4c30e975a4326a208053a48fc04fb, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:26.811631 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d175f18-28db-4bd5-9e2e-8078fbb76010 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:26.825076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1054/2096] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:26 2022] PUT /identity/v3/projects/3487d13d285441b6888deb400ca9122c/users/a1e4c30e975a4326a208053a48fc04fb/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:26.829206 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a0441-0fe9-4999-92f3-1be725a731ae admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:26.839619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a0441-0fe9-4999-92f3-1be725a731ae admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:26.859090 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:26.898147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fda861-9335-4827-a47c-39aea06ecdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:26.898786 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fda861-9335-4827-a47c-39aea06ecdce admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:26.899064 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fda861-9335-4827-a47c-39aea06ecdce admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:26.899331 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fda861-9335-4827-a47c-39aea06ecdce admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:26.899965 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fda861-9335-4827-a47c-39aea06ecdce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:26.901608 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fda861-9335-4827-a47c-39aea06ecdce admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:26.911261 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1043/2097] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:26.915487 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d175f18-28db-4bd5-9e2e-8078fbb76010 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:26.921052 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:26.927436 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d175f18-28db-4bd5-9e2e-8078fbb76010 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:26.933600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6116d-ef98-4ac2-b664-51aaf2af7068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:26.934304 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6116d-ef98-4ac2-b664-51aaf2af7068 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:26.934647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6116d-ef98-4ac2-b664-51aaf2af7068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:26.934974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6116d-ef98-4ac2-b664-51aaf2af7068 admin admin] PATH_INFO: `/v3/projects/3487d13d285441b6888deb400ca9122c/users/a1e4c30e975a4326a208053a48fc04fb/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:26.937023 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6116d-ef98-4ac2-b664-51aaf2af7068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3487d13d285441b6888deb400ca9122c, user_id=a1e4c30e975a4326a208053a48fc04fb, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:26.939724 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6116d-ef98-4ac2-b664-51aaf2af7068 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:26.951162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1055/2098] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:26 2022] PUT /identity/v3/projects/3487d13d285441b6888deb400ca9122c/users/a1e4c30e975a4326a208053a48fc04fb/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:26.962043 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e3c93-d05a-4994-b6bd-fcaa1d5c57ee None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:26.962331 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e3c93-d05a-4994-b6bd-fcaa1d5c57ee None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:26.962611 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e3c93-d05a-4994-b6bd-fcaa1d5c57ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:26.989650 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-566e3c93-d05a-4994-b6bd-fcaa1d5c57ee None None] MFA Rules not processed for user `a1e4c30e975a4326a208053a48fc04fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:26.990853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-566e3c93-d05a-4994-b6bd-fcaa1d5c57ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:27.060206 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1044/2099] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:27.072601 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6116d-ef98-4ac2-b664-51aaf2af7068 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:27.074684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-53a6116d-ef98-4ac2-b664-51aaf2af7068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:27.153957 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6116d-ef98-4ac2-b664-51aaf2af7068 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:27.158897 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628f35e-3dbe-4248-8848-c11700a4eb55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:27.159628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628f35e-3dbe-4248-8848-c11700a4eb55 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:27.159912 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628f35e-3dbe-4248-8848-c11700a4eb55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:27.160216 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628f35e-3dbe-4248-8848-c11700a4eb55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:27.170849 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628f35e-3dbe-4248-8848-c11700a4eb55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:27.172590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628f35e-3dbe-4248-8848-c11700a4eb55 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:27.188843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1056/2100] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:58:27.279590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e3c93-d05a-4994-b6bd-fcaa1d5c57ee None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:27.291032 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e3c93-d05a-4994-b6bd-fcaa1d5c57ee None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:27.295076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7850503-1978-466b-82a5-324e1bf16764 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:27.295743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7850503-1978-466b-82a5-324e1bf16764 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:27.296024 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7850503-1978-466b-82a5-324e1bf16764 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:27.296327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7850503-1978-466b-82a5-324e1bf16764 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:27.365062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7850503-1978-466b-82a5-324e1bf16764 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:27.366630 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7850503-1978-466b-82a5-324e1bf16764 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:27.376770 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1045/2101] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:31.786250 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66693f-d939-4231-b9c6-5c0234cc0af3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:31.786906 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66693f-d939-4231-b9c6-5c0234cc0af3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:31.786906 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66693f-d939-4231-b9c6-5c0234cc0af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:31.806687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5e66693f-d939-4231-b9c6-5c0234cc0af3 None None] MFA Rules not processed for user `b11abed8d4424a1e9957c3dc9aefadd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:31.807985 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5e66693f-d939-4231-b9c6-5c0234cc0af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:31.872856 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1057/2102] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 24 08:58:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:31.923763 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7850503-1978-466b-82a5-324e1bf16764 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:31.925368 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f7850503-1978-466b-82a5-324e1bf16764 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:31.975971 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:32.000852 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7850503-1978-466b-82a5-324e1bf16764 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:32.005998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949006f-a3c5-4d35-8362-84c191a2b370 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46b1417d4ac24c8982b2052f4e99042a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:32.006616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949006f-a3c5-4d35-8362-84c191a2b370 service swift] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:32.006890 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949006f-a3c5-4d35-8362-84c191a2b370 service swift] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:32.007159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949006f-a3c5-4d35-8362-84c191a2b370 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:32.017019 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949006f-a3c5-4d35-8362-84c191a2b370 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:32.018822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949006f-a3c5-4d35-8362-84c191a2b370 service swift] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:32.030605 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1046/2103] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 24 08:58:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:58:41.861297 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=55,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:42.026612 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=12 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:48.507727 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66693f-d939-4231-b9c6-5c0234cc0af3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:48.512877 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5e66693f-d939-4231-b9c6-5c0234cc0af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:48.597117 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66693f-d939-4231-b9c6-5c0234cc0af3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:48.605586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6436e-9cf4-476c-9e67-1a9999a00916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:48.606289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6436e-9cf4-476c-9e67-1a9999a00916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:48.606770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6436e-9cf4-476c-9e67-1a9999a00916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:48.607229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6436e-9cf4-476c-9e67-1a9999a00916 admin admin] PATH_INFO: `/v3/users/0212144e820f4724ab36f683d6448794` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:48.608698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6436e-9cf4-476c-9e67-1a9999a00916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0212144e820f4724ab36f683d6448794)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:48.610825 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6436e-9cf4-476c-9e67-1a9999a00916 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:48.629781 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949006f-a3c5-4d35-8362-84c191a2b370 service swift] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:48.642890 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949006f-a3c5-4d35-8362-84c191a2b370 service swift] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:48.650620 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c30c1c-6093-49d1-ae5f-3cd40986b61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:48.651343 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c30c1c-6093-49d1-ae5f-3cd40986b61b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:48.651709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c30c1c-6093-49d1-ae5f-3cd40986b61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:48.651981 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c30c1c-6093-49d1-ae5f-3cd40986b61b admin admin] PATH_INFO: `/v3/users/b913bfe9ba934806a4f17551d5dbbfea` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:48.653507 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c30c1c-6093-49d1-ae5f-3cd40986b61b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b913bfe9ba934806a4f17551d5dbbfea)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:48.657473 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c30c1c-6093-49d1-ae5f-3cd40986b61b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:48.676131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-58c6436e-9cf4-476c-9e67-1a9999a00916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0212144e820f4724ab36f683d6448794'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:48.677098 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-58c6436e-9cf4-476c-9e67-1a9999a00916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0212144e820f4724ab36f683d6448794'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:48.686195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-58c6436e-9cf4-476c-9e67-1a9999a00916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0212144e820f4724ab36f683d6448794'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:48.696155 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-58c6436e-9cf4-476c-9e67-1a9999a00916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0212144e820f4724ab36f683d6448794'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:48.711936 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-58c6436e-9cf4-476c-9e67-1a9999a00916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0212144e820f4724ab36f683d6448794'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:48.721322 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e8c30c1c-6093-49d1-ae5f-3cd40986b61b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b913bfe9ba934806a4f17551d5dbbfea'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:48.724354 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1058/2104] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:48 2022] DELETE /identity/v3/users/0212144e820f4724ab36f683d6448794 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:48.726247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a10ba5-0adf-4343-9c9b-619a97997924 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:48.726682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a10ba5-0adf-4343-9c9b-619a97997924 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:48.727112 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a10ba5-0adf-4343-9c9b-619a97997924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:48.730467 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e8c30c1c-6093-49d1-ae5f-3cd40986b61b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b913bfe9ba934806a4f17551d5dbbfea'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:48.748133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-45a10ba5-0adf-4343-9c9b-619a97997924 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:48.749363 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-45a10ba5-0adf-4343-9c9b-619a97997924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:48.751050 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e8c30c1c-6093-49d1-ae5f-3cd40986b61b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b913bfe9ba934806a4f17551d5dbbfea'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:48.761833 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e8c30c1c-6093-49d1-ae5f-3cd40986b61b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b913bfe9ba934806a4f17551d5dbbfea'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:48.762979 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e8c30c1c-6093-49d1-ae5f-3cd40986b61b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b913bfe9ba934806a4f17551d5dbbfea'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:48.770311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1047/2105] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:48 2022] DELETE /identity/v3/users/b913bfe9ba934806a4f17551d5dbbfea => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:48.775908 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d35af-5c1f-43be-a679-29af6455cb02 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:48.776479 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d35af-5c1f-43be-a679-29af6455cb02 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:48.776829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d35af-5c1f-43be-a679-29af6455cb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:48.814281 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-a98d35af-5c1f-43be-a679-29af6455cb02 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:48.815366 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a98d35af-5c1f-43be-a679-29af6455cb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:48.823714 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1059/2106] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:48.834456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a10ba5-0adf-4343-9c9b-619a97997924 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:48.836561 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-45a10ba5-0adf-4343-9c9b-619a97997924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:48.860234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:48.920976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a10ba5-0adf-4343-9c9b-619a97997924 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:48.932186 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1048/2107] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:48.934860 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d35af-5c1f-43be-a679-29af6455cb02 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:48.999373 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cbf93-88f9-4be5-a2c7-8f0143f0997e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:49.000001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cbf93-88f9-4be5-a2c7-8f0143f0997e service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:49.000803 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cbf93-88f9-4be5-a2c7-8f0143f0997e service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:49.001080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cbf93-88f9-4be5-a2c7-8f0143f0997e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:49.018777 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d35af-5c1f-43be-a679-29af6455cb02 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:49.023116 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8cbf93-88f9-4be5-a2c7-8f0143f0997e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:49.023372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80746297-59ee-4730-90d4-73c1a8b55588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:49.023862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80746297-59ee-4730-90d4-73c1a8b55588 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:49.024139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80746297-59ee-4730-90d4-73c1a8b55588 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:49.024416 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80746297-59ee-4730-90d4-73c1a8b55588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:49.027688 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8cbf93-88f9-4be5-a2c7-8f0143f0997e service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:49.045872 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80746297-59ee-4730-90d4-73c1a8b55588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:49.049481 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80746297-59ee-4730-90d4-73c1a8b55588 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:49.070640 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1060/2108] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:49.078056 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1049/2109] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:58:49.171891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40651bd9-ec85-4bc5-9903-c383f13ba76c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:49.172266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40651bd9-ec85-4bc5-9903-c383f13ba76c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:49.172542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40651bd9-ec85-4bc5-9903-c383f13ba76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:49.206767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-40651bd9-ec85-4bc5-9903-c383f13ba76c None None] MFA Rules not processed for user `b11abed8d4424a1e9957c3dc9aefadd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:49.208362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-40651bd9-ec85-4bc5-9903-c383f13ba76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:49.298848 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1061/2110] 10.222.0.67 () {58 vars in 967 bytes} [Wed Aug 24 08:58:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:49.321097 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80746297-59ee-4730-90d4-73c1a8b55588 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:49.323653 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-80746297-59ee-4730-90d4-73c1a8b55588 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:49.350469 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:49.440669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80746297-59ee-4730-90d4-73c1a8b55588 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:49.446694 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590101e-0067-49f1-84a9-ad86868fe6c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46b1417d4ac24c8982b2052f4e99042a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:49.448031 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590101e-0067-49f1-84a9-ad86868fe6c2 service swift] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:49.448340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590101e-0067-49f1-84a9-ad86868fe6c2 service swift] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:49.448612 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590101e-0067-49f1-84a9-ad86868fe6c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:49.459444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590101e-0067-49f1-84a9-ad86868fe6c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:49.461409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590101e-0067-49f1-84a9-ad86868fe6c2 service swift] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:49.473074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1050/2111] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 24 08:58:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:58:49.627853 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40651bd9-ec85-4bc5-9903-c383f13ba76c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:49.629770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-40651bd9-ec85-4bc5-9903-c383f13ba76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:49.644371 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590101e-0067-49f1-84a9-ad86868fe6c2 service swift] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:49.646972 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3590101e-0067-49f1-84a9-ad86868fe6c2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:49.686959 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40651bd9-ec85-4bc5-9903-c383f13ba76c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:49.691640 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590101e-0067-49f1-84a9-ad86868fe6c2 service swift] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:49.702164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c692797-6062-4f49-9b5b-b2af21bfabb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:49.707470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfd50b-d233-4df1-8d50-e5a6aeabbd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:49.708545 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfd50b-d233-4df1-8d50-e5a6aeabbd90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:49.709075 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfd50b-d233-4df1-8d50-e5a6aeabbd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:49.709453 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfd50b-d233-4df1-8d50-e5a6aeabbd90 admin admin] PATH_INFO: `/v3/projects/8b9640d286cd4b1c86958a4cedc3e1a0` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:49.711592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdfd50b-d233-4df1-8d50-e5a6aeabbd90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b9640d286cd4b1c86958a4cedc3e1a0)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:49.716909 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c692797-6062-4f49-9b5b-b2af21bfabb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:49.717717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c692797-6062-4f49-9b5b-b2af21bfabb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:49.718004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c692797-6062-4f49-9b5b-b2af21bfabb3 admin admin] PATH_INFO: `/v3/projects/993f03f0e9344ee7b44767018ba3fab1` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:49.722388 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c692797-6062-4f49-9b5b-b2af21bfabb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=993f03f0e9344ee7b44767018ba3fab1)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:49.723926 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdfd50b-d233-4df1-8d50-e5a6aeabbd90 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:49.724676 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c692797-6062-4f49-9b5b-b2af21bfabb3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:49.800974 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-afdfd50b-d233-4df1-8d50-e5a6aeabbd90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b9640d286cd4b1c86958a4cedc3e1a0', 'request_id': 'req-afdfd50b-d233-4df1-8d50-e5a6aeabbd90'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:49.802183 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1c692797-6062-4f49-9b5b-b2af21bfabb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '993f03f0e9344ee7b44767018ba3fab1', 'request_id': 'req-1c692797-6062-4f49-9b5b-b2af21bfabb3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:49.814398 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-afdfd50b-d233-4df1-8d50-e5a6aeabbd90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b9640d286cd4b1c86958a4cedc3e1a0', 'request_id': 'req-afdfd50b-d233-4df1-8d50-e5a6aeabbd90'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:49.818394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1c692797-6062-4f49-9b5b-b2af21bfabb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '993f03f0e9344ee7b44767018ba3fab1', 'request_id': 'req-1c692797-6062-4f49-9b5b-b2af21bfabb3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:49.827627 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-afdfd50b-d233-4df1-8d50-e5a6aeabbd90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b9640d286cd4b1c86958a4cedc3e1a0', 'request_id': 'req-afdfd50b-d233-4df1-8d50-e5a6aeabbd90'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:49.833087 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1c692797-6062-4f49-9b5b-b2af21bfabb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '993f03f0e9344ee7b44767018ba3fab1', 'request_id': 'req-1c692797-6062-4f49-9b5b-b2af21bfabb3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:49.846970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-afdfd50b-d233-4df1-8d50-e5a6aeabbd90 admin admin] The token cache is being invalidate because project 8b9640d286cd4b1c86958a4cedc3e1a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:49.847275 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-afdfd50b-d233-4df1-8d50-e5a6aeabbd90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:49.847972 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-afdfd50b-d233-4df1-8d50-e5a6aeabbd90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:49.849857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1051/2112] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:49 2022] DELETE /identity/v3/projects/8b9640d286cd4b1c86958a4cedc3e1a0 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:49.857964 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687b383-8497-4eb0-8061-10c168ceebde None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:49.858334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687b383-8497-4eb0-8061-10c168ceebde None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:49.858683 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687b383-8497-4eb0-8061-10c168ceebde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:49.863644 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1c692797-6062-4f49-9b5b-b2af21bfabb3 admin admin] The token cache is being invalidate because project 993f03f0e9344ee7b44767018ba3fab1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:49.864032 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1c692797-6062-4f49-9b5b-b2af21bfabb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:49.865003 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1c692797-6062-4f49-9b5b-b2af21bfabb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:49.867403 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1062/2113] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:49 2022] DELETE /identity/v3/projects/993f03f0e9344ee7b44767018ba3fab1 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:49.872277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c692797-6062-4f49-9b5b-b2af21bfabb3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:49.876068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1c692797-6062-4f49-9b5b-b2af21bfabb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:49.888222 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-8687b383-8497-4eb0-8061-10c168ceebde None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:49.889460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8687b383-8497-4eb0-8061-10c168ceebde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:49.894144 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:50.001798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c692797-6062-4f49-9b5b-b2af21bfabb3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:50.002904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1052/2114] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:50.008021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b2e8b-4a20-4ee8-8e53-e87e8f354ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:50.008756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b2e8b-4a20-4ee8-8e53-e87e8f354ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:50.009058 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b2e8b-4a20-4ee8-8e53-e87e8f354ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:50.009324 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b2e8b-4a20-4ee8-8e53-e87e8f354ba8 admin admin] PATH_INFO: `/v3/users/a1e4c30e975a4326a208053a48fc04fb` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:50.010953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80438e7c-80aa-469c-8ba6-2d0d71c2c5d6 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:50.011238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80438e7c-80aa-469c-8ba6-2d0d71c2c5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:50.011504 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80438e7c-80aa-469c-8ba6-2d0d71c2c5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:50.013627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b2e8b-4a20-4ee8-8e53-e87e8f354ba8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1e4c30e975a4326a208053a48fc04fb)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:50.015719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b2e8b-4a20-4ee8-8e53-e87e8f354ba8 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:50.036189 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-80438e7c-80aa-469c-8ba6-2d0d71c2c5d6 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:50.037543 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-80438e7c-80aa-469c-8ba6-2d0d71c2c5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:50.048641 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1053/2115] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:50.057823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80438e7c-80aa-469c-8ba6-2d0d71c2c5d6 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:50.075875 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80438e7c-80aa-469c-8ba6-2d0d71c2c5d6 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:50.084352 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963d822-bcba-40f4-8096-5d873d87a193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:50.085742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963d822-bcba-40f4-8096-5d873d87a193 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:50.086202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963d822-bcba-40f4-8096-5d873d87a193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:50.093736 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-855b2e8b-4a20-4ee8-8e53-e87e8f354ba8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1e4c30e975a4326a208053a48fc04fb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:50.095033 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-855b2e8b-4a20-4ee8-8e53-e87e8f354ba8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1e4c30e975a4326a208053a48fc04fb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:50.096460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963d822-bcba-40f4-8096-5d873d87a193 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:50.100744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963d822-bcba-40f4-8096-5d873d87a193 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:50.101350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963d822-bcba-40f4-8096-5d873d87a193 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:50.104784 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963d822-bcba-40f4-8096-5d873d87a193 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:50.107887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-855b2e8b-4a20-4ee8-8e53-e87e8f354ba8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1e4c30e975a4326a208053a48fc04fb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:50.138823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1054/2116] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:58:50.139578 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-855b2e8b-4a20-4ee8-8e53-e87e8f354ba8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1e4c30e975a4326a208053a48fc04fb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:50.140298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963d822-bcba-40f4-8096-5d873d87a193 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:50.144177 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e963d822-bcba-40f4-8096-5d873d87a193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:50.158180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-855b2e8b-4a20-4ee8-8e53-e87e8f354ba8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1e4c30e975a4326a208053a48fc04fb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:50.162955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963d822-bcba-40f4-8096-5d873d87a193 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:50.167202 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1063/2117] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:49 2022] DELETE /identity/v3/users/a1e4c30e975a4326a208053a48fc04fb => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:50.169089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f80482-c0d5-47dc-a061-8bb129c46854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:50.169705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f80482-c0d5-47dc-a061-8bb129c46854 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:50.170014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f80482-c0d5-47dc-a061-8bb129c46854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:50.170279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f80482-c0d5-47dc-a061-8bb129c46854 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:50.171264 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f80482-c0d5-47dc-a061-8bb129c46854 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:50.172899 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f80482-c0d5-47dc-a061-8bb129c46854 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:50.179395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:50.223187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1055/2118] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:50 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:50.225921 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b2e8b-4a20-4ee8-8e53-e87e8f354ba8 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:50.328860 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b2e8b-4a20-4ee8-8e53-e87e8f354ba8 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:50.376004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:50.417224 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2c028-30f2-43e8-a352-cf1babdb1e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:50.418708 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2c028-30f2-43e8-a352-cf1babdb1e15 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:50.419061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2c028-30f2-43e8-a352-cf1babdb1e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:50.419495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2c028-30f2-43e8-a352-cf1babdb1e15 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:50.420196 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e2c028-30f2-43e8-a352-cf1babdb1e15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:50.422357 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e2c028-30f2-43e8-a352-cf1babdb1e15 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:50.452896 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1064/2119] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:50 2022] POST /identity/v3/users => generated 287 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:50.457455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f80482-c0d5-47dc-a061-8bb129c46854 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:50.472450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f80482-c0d5-47dc-a061-8bb129c46854 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:50.481097 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abc8d9-2e8c-472c-b895-cb9ef09285c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:50.484372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abc8d9-2e8c-472c-b895-cb9ef09285c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:50.484661 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abc8d9-2e8c-472c-b895-cb9ef09285c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:50.485006 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abc8d9-2e8c-472c-b895-cb9ef09285c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:50.485880 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34abc8d9-2e8c-472c-b895-cb9ef09285c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:50.487651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34abc8d9-2e8c-472c-b895-cb9ef09285c9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:50.499197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1056/2120] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:50.503302 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2c028-30f2-43e8-a352-cf1babdb1e15 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:50.514927 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2c028-30f2-43e8-a352-cf1babdb1e15 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:50.519977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c4c81-acd3-4d2a-a8de-875d94192cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:50.520645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c4c81-acd3-4d2a-a8de-875d94192cfe admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:50.520948 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c4c81-acd3-4d2a-a8de-875d94192cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:50.521227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c4c81-acd3-4d2a-a8de-875d94192cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:50.522077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c4c81-acd3-4d2a-a8de-875d94192cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:50.523693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c4c81-acd3-4d2a-a8de-875d94192cfe admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:50.536686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1065/2121] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:50.540478 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abc8d9-2e8c-472c-b895-cb9ef09285c9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:50.551802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abc8d9-2e8c-472c-b895-cb9ef09285c9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:50.558085 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3b645-632e-4669-bd03-660514a2c5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:50.559580 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3b645-632e-4669-bd03-660514a2c5b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:50.560010 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3b645-632e-4669-bd03-660514a2c5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:50.560329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3b645-632e-4669-bd03-660514a2c5b8 admin admin] PATH_INFO: `/v3/projects/080318594f144fa3852577bc02ee5357/users/46b3aa99a5e84ef2a8df193628feb0d6/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:50.585234 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3b645-632e-4669-bd03-660514a2c5b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=080318594f144fa3852577bc02ee5357, user_id=46b3aa99a5e84ef2a8df193628feb0d6, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:50.585536 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c4c81-acd3-4d2a-a8de-875d94192cfe admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:50.589545 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4f8c4c81-acd3-4d2a-a8de-875d94192cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:50.590371 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3b645-632e-4669-bd03-660514a2c5b8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:50.603828 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c4c81-acd3-4d2a-a8de-875d94192cfe admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:50.612259 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1057/2122] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:50 2022] PUT /identity/v3/projects/080318594f144fa3852577bc02ee5357/users/46b3aa99a5e84ef2a8df193628feb0d6/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:50.617700 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01ad01-546d-4516-b33d-f51bdee64039 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:50.618570 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01ad01-546d-4516-b33d-f51bdee64039 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:50.619045 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01ad01-546d-4516-b33d-f51bdee64039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:50.642477 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-0f01ad01-546d-4516-b33d-f51bdee64039 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:50.644383 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0f01ad01-546d-4516-b33d-f51bdee64039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:50.682185 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675a4f6-8bb5-4f94-a22e-44f67fea3ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:50.683168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675a4f6-8bb5-4f94-a22e-44f67fea3ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:50.683367 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675a4f6-8bb5-4f94-a22e-44f67fea3ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:50.683652 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675a4f6-8bb5-4f94-a22e-44f67fea3ce4 admin admin] PATH_INFO: `/v3/projects/3487d13d285441b6888deb400ca9122c` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:50.685308 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6675a4f6-8bb5-4f94-a22e-44f67fea3ce4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3487d13d285441b6888deb400ca9122c)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:50.687570 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6675a4f6-8bb5-4f94-a22e-44f67fea3ce4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:50.722200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1058/2123] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:50.733831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01ad01-546d-4516-b33d-f51bdee64039 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:50.735194 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0f01ad01-546d-4516-b33d-f51bdee64039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:50.758445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6675a4f6-8bb5-4f94-a22e-44f67fea3ce4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3487d13d285441b6888deb400ca9122c', 'request_id': 'req-6675a4f6-8bb5-4f94-a22e-44f67fea3ce4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:50.766276 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6675a4f6-8bb5-4f94-a22e-44f67fea3ce4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3487d13d285441b6888deb400ca9122c', 'request_id': 'req-6675a4f6-8bb5-4f94-a22e-44f67fea3ce4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:50.777714 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6675a4f6-8bb5-4f94-a22e-44f67fea3ce4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3487d13d285441b6888deb400ca9122c', 'request_id': 'req-6675a4f6-8bb5-4f94-a22e-44f67fea3ce4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:50.792540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6675a4f6-8bb5-4f94-a22e-44f67fea3ce4 admin admin] The token cache is being invalidate because project 3487d13d285441b6888deb400ca9122c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:50.792855 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6675a4f6-8bb5-4f94-a22e-44f67fea3ce4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:50.793625 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6675a4f6-8bb5-4f94-a22e-44f67fea3ce4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:50.795477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1066/2124] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:50 2022] DELETE /identity/v3/projects/3487d13d285441b6888deb400ca9122c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:50.800372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36bf67-8508-4c8e-9257-357cd8c03baa None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:50.800664 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36bf67-8508-4c8e-9257-357cd8c03baa None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:50.800930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36bf67-8508-4c8e-9257-357cd8c03baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:50.823045 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01ad01-546d-4516-b33d-f51bdee64039 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:50.836003 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8f36bf67-8508-4c8e-9257-357cd8c03baa None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:50.838357 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8f36bf67-8508-4c8e-9257-357cd8c03baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:50.906468 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:50.913525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a3a84-3bee-4a9e-9fa2-33888e7f5d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:50.914145 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a3a84-3bee-4a9e-9fa2-33888e7f5d11 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:50.914565 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a3a84-3bee-4a9e-9fa2-33888e7f5d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:50.915057 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a3a84-3bee-4a9e-9fa2-33888e7f5d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:50.917538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4b6a3a84-3bee-4a9e-9fa2-33888e7f5d11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:50.934122 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1067/2125] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:50.941078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:50.944076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab75b0f-3468-4bc0-9246-9392dc276540 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:50.944406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab75b0f-3468-4bc0-9246-9392dc276540 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:50.944532 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab75b0f-3468-4bc0-9246-9392dc276540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:50.966035 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5ab75b0f-3468-4bc0-9246-9392dc276540 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:50.967251 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5ab75b0f-3468-4bc0-9246-9392dc276540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:50.976163 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1068/2126] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:50.981238 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab75b0f-3468-4bc0-9246-9392dc276540 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:50.990230 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a3a84-3bee-4a9e-9fa2-33888e7f5d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:50.990802 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab75b0f-3468-4bc0-9246-9392dc276540 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:50.991716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a3a84-3bee-4a9e-9fa2-33888e7f5d11 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:50.996001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a47cb3-d882-4a4e-8c5f-db94d14f1904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:50.996645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a47cb3-d882-4a4e-8c5f-db94d14f1904 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:50.996919 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a47cb3-d882-4a4e-8c5f-db94d14f1904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:50.997194 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a47cb3-d882-4a4e-8c5f-db94d14f1904 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:50.997849 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a47cb3-d882-4a4e-8c5f-db94d14f1904 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:50.998150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a47cb3-d882-4a4e-8c5f-db94d14f1904 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:50.999749 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a47cb3-d882-4a4e-8c5f-db94d14f1904 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:51.002398 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1059/2127] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:51.016151 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1069/2128] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:58:51.017584 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a3a84-3bee-4a9e-9fa2-33888e7f5d11 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:51.028794 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a3a84-3bee-4a9e-9fa2-33888e7f5d11 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:51.034885 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ca686-1d00-48a3-8c83-5b71dfce0b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:51.035551 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ca686-1d00-48a3-8c83-5b71dfce0b71 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:51.035928 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ca686-1d00-48a3-8c83-5b71dfce0b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:51.036221 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ca686-1d00-48a3-8c83-5b71dfce0b71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:51.036931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ca686-1d00-48a3-8c83-5b71dfce0b71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:51.038880 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ca686-1d00-48a3-8c83-5b71dfce0b71 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:51.069842 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1060/2129] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:51 2022] POST /identity/v3/projects => generated 409 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:51.074440 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a47cb3-d882-4a4e-8c5f-db94d14f1904 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:51.085106 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a47cb3-d882-4a4e-8c5f-db94d14f1904 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:51.164562 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29cf70e-3d3a-4442-b6fb-6dc6e049f561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:51.166738 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29cf70e-3d3a-4442-b6fb-6dc6e049f561 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:51.167218 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29cf70e-3d3a-4442-b6fb-6dc6e049f561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:51.167501 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29cf70e-3d3a-4442-b6fb-6dc6e049f561 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:51.168416 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29cf70e-3d3a-4442-b6fb-6dc6e049f561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:51.170494 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29cf70e-3d3a-4442-b6fb-6dc6e049f561 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:51.204651 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1070/2130] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:51 2022] POST /identity/v3/users => generated 305 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:51.206033 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ca686-1d00-48a3-8c83-5b71dfce0b71 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:51.222432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ca686-1d00-48a3-8c83-5b71dfce0b71 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:51.228997 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20a557-8a29-4123-934d-23a02841e52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:51.230201 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20a557-8a29-4123-934d-23a02841e52e admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:51.230482 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20a557-8a29-4123-934d-23a02841e52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:51.231303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20a557-8a29-4123-934d-23a02841e52e admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:51.231974 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20a557-8a29-4123-934d-23a02841e52e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:51.234026 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20a557-8a29-4123-934d-23a02841e52e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:51.249011 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1061/2131] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:51.250246 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29cf70e-3d3a-4442-b6fb-6dc6e049f561 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:51.266362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29cf70e-3d3a-4442-b6fb-6dc6e049f561 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:51.271386 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa8625-2144-4380-a435-4b28640b9c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:51.272217 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa8625-2144-4380-a435-4b28640b9c79 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:51.272504 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa8625-2144-4380-a435-4b28640b9c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:51.272772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa8625-2144-4380-a435-4b28640b9c79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:51.274075 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa8625-2144-4380-a435-4b28640b9c79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:51.276756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa8625-2144-4380-a435-4b28640b9c79 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:51.293999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1071/2132] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:51.301793 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20a557-8a29-4123-934d-23a02841e52e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:51.313102 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20a557-8a29-4123-934d-23a02841e52e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:51.318019 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57f838-c72a-4ceb-aeed-617f19a540dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:51.318825 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57f838-c72a-4ceb-aeed-617f19a540dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:51.319209 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57f838-c72a-4ceb-aeed-617f19a540dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:51.319583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57f838-c72a-4ceb-aeed-617f19a540dd admin admin] PATH_INFO: `/v3/projects/5d08444163c544d58245c44f97d74f4a/users/f97a11029ba543c286048f3f6e934324/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:51.336699 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57f838-c72a-4ceb-aeed-617f19a540dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d08444163c544d58245c44f97d74f4a, user_id=f97a11029ba543c286048f3f6e934324, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:51.342127 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57f838-c72a-4ceb-aeed-617f19a540dd admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:51.364520 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1062/2133] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:51 2022] PUT /identity/v3/projects/5d08444163c544d58245c44f97d74f4a/users/f97a11029ba543c286048f3f6e934324/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:51.378892 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2d2a3-086b-421f-b4e5-6e44e0615e9f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:51.379590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2d2a3-086b-421f-b4e5-6e44e0615e9f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:51.379957 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2d2a3-086b-421f-b4e5-6e44e0615e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:51.427470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-94a2d2a3-086b-421f-b4e5-6e44e0615e9f None None] MFA Rules not processed for user `f97a11029ba543c286048f3f6e934324`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:51.428754 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-94a2d2a3-086b-421f-b4e5-6e44e0615e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:51.530006 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1072/2134] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:51 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:51.534868 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57f838-c72a-4ceb-aeed-617f19a540dd admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:51.609046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57f838-c72a-4ceb-aeed-617f19a540dd admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:51.615454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6fe201-12c4-4cec-9c19-cc338429ddef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:51.616368 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fe201-12c4-4cec-9c19-cc338429ddef admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:51.616653 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fe201-12c4-4cec-9c19-cc338429ddef admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:51.616929 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fe201-12c4-4cec-9c19-cc338429ddef admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:51.617571 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6fe201-12c4-4cec-9c19-cc338429ddef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:51.620130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6fe201-12c4-4cec-9c19-cc338429ddef admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:51.651819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1063/2135] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:51 2022] POST /identity/v3/projects => generated 409 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:51.655967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2d2a3-086b-421f-b4e5-6e44e0615e9f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:51.669105 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2d2a3-086b-421f-b4e5-6e44e0615e9f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:51.702402 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:51.751414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c3014-c964-49fd-acf8-b5f9b5ee8bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:51.752041 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c3014-c964-49fd-acf8-b5f9b5ee8bba admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:51.752371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c3014-c964-49fd-acf8-b5f9b5ee8bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:51.752646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c3014-c964-49fd-acf8-b5f9b5ee8bba admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:51.753290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c3014-c964-49fd-acf8-b5f9b5ee8bba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:51.755643 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c3014-c964-49fd-acf8-b5f9b5ee8bba admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:51.781350 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1073/2136] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:51 2022] POST /identity/v3/users => generated 311 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:58:51.786313 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6fe201-12c4-4cec-9c19-cc338429ddef admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:51.798394 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6fe201-12c4-4cec-9c19-cc338429ddef admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:51.807141 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9140c5b-3b0b-4264-918d-3b2d8b34bb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:51.808817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9140c5b-3b0b-4264-918d-3b2d8b34bb80 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:51.809162 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9140c5b-3b0b-4264-918d-3b2d8b34bb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:51.809630 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9140c5b-3b0b-4264-918d-3b2d8b34bb80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:51.810312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9140c5b-3b0b-4264-918d-3b2d8b34bb80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:51.812225 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9140c5b-3b0b-4264-918d-3b2d8b34bb80 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:51.826609 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1064/2137] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:58:51.831982 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c3014-c964-49fd-acf8-b5f9b5ee8bba admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:51.844246 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c3014-c964-49fd-acf8-b5f9b5ee8bba admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:51.849050 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffcde4-49a2-4961-bdeb-6f1651ea1d5b None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:51.849838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffcde4-49a2-4961-bdeb-6f1651ea1d5b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:51.850486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffcde4-49a2-4961-bdeb-6f1651ea1d5b None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:51.851602 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409a563-6939-4f5a-abde-ced64f4d082d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:51.852325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409a563-6939-4f5a-abde-ced64f4d082d admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:51.852610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409a563-6939-4f5a-abde-ced64f4d082d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:51.854380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1065/2138] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 24 08:58:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 08:58:51.856635 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409a563-6939-4f5a-abde-ced64f4d082d admin admin] PATH_INFO: `/v3/projects/b0f39107b8594d97b64121b5b2a011f2/users/794a266461694f8bacc61ba4ce613ec9/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:51.876448 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409a563-6939-4f5a-abde-ced64f4d082d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f39107b8594d97b64121b5b2a011f2, user_id=794a266461694f8bacc61ba4ce613ec9, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:51.880801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83af3f2-fc47-4f97-b6ea-27effcffbb0e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:51.881879 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83af3f2-fc47-4f97-b6ea-27effcffbb0e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:51.881879 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83af3f2-fc47-4f97-b6ea-27effcffbb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:51.884612 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409a563-6939-4f5a-abde-ced64f4d082d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:51.905039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f83af3f2-fc47-4f97-b6ea-27effcffbb0e None None] MFA Rules not processed for user `33a0844a1c8f4ea4932e12d53e5ee9ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:51.905877 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1074/2139] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:51 2022] PUT /identity/v3/projects/b0f39107b8594d97b64121b5b2a011f2/users/794a266461694f8bacc61ba4ce613ec9/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:51.906552 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f83af3f2-fc47-4f97-b6ea-27effcffbb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:51.913570 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409a563-6939-4f5a-abde-ced64f4d082d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:51.946171 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:52.019292 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1066/2140] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 24 08:58:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:52.020569 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83af3f2-fc47-4f97-b6ea-27effcffbb0e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:52.022617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f83af3f2-fc47-4f97-b6ea-27effcffbb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:52.026276 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409a563-6939-4f5a-abde-ced64f4d082d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:52.032552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e90d0-25eb-4403-80f8-69e4fc4cbd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:52.033605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e90d0-25eb-4403-80f8-69e4fc4cbd70 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:52.034158 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e90d0-25eb-4403-80f8-69e4fc4cbd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:52.034640 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e90d0-25eb-4403-80f8-69e4fc4cbd70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:52.035522 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e90d0-25eb-4403-80f8-69e4fc4cbd70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:52.038660 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e90d0-25eb-4403-80f8-69e4fc4cbd70 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:52.046494 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83af3f2-fc47-4f97-b6ea-27effcffbb0e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:52.052673 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1075/2141] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:58:52.057722 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e90d0-25eb-4403-80f8-69e4fc4cbd70 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:52.059220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8481e-0c4e-4255-b2a5-5508b0937359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:52.060753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8481e-0c4e-4255-b2a5-5508b0937359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:52.061039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8481e-0c4e-4255-b2a5-5508b0937359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:52.061319 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8481e-0c4e-4255-b2a5-5508b0937359 admin admin] PATH_INFO: `/v3/users/3095e7f94a58497c812890fa86e08eec` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:52.064077 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8481e-0c4e-4255-b2a5-5508b0937359 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3095e7f94a58497c812890fa86e08eec)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:52.067918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8481e-0c4e-4255-b2a5-5508b0937359 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:52.069717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e90d0-25eb-4403-80f8-69e4fc4cbd70 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:52.084429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ee16b-7ddc-4f3a-a948-7599d0658c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:52.085415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ee16b-7ddc-4f3a-a948-7599d0658c1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:52.085933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ee16b-7ddc-4f3a-a948-7599d0658c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:52.086445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ee16b-7ddc-4f3a-a948-7599d0658c1a admin admin] PATH_INFO: `/v3/projects/b0f39107b8594d97b64121b5b2a011f2/users/794a266461694f8bacc61ba4ce613ec9/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:52.089079 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ee16b-7ddc-4f3a-a948-7599d0658c1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f39107b8594d97b64121b5b2a011f2, user_id=794a266461694f8bacc61ba4ce613ec9, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:52.092658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ee16b-7ddc-4f3a-a948-7599d0658c1a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:52.112212 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1076/2142] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:52 2022] PUT /identity/v3/projects/b0f39107b8594d97b64121b5b2a011f2/users/794a266461694f8bacc61ba4ce613ec9/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:52.113310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ee16b-7ddc-4f3a-a948-7599d0658c1a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:52.120912 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-570ee16b-7ddc-4f3a-a948-7599d0658c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:52.154019 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e5d8481e-0c4e-4255-b2a5-5508b0937359 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3095e7f94a58497c812890fa86e08eec'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:52.164797 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e5d8481e-0c4e-4255-b2a5-5508b0937359 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3095e7f94a58497c812890fa86e08eec'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:52.189431 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e5d8481e-0c4e-4255-b2a5-5508b0937359 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3095e7f94a58497c812890fa86e08eec'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:52.204564 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e5d8481e-0c4e-4255-b2a5-5508b0937359 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3095e7f94a58497c812890fa86e08eec'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:52.205335 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e5d8481e-0c4e-4255-b2a5-5508b0937359 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3095e7f94a58497c812890fa86e08eec'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:52.214393 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1067/2143] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:58:52 2022] DELETE /identity/v3/users/3095e7f94a58497c812890fa86e08eec => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:52.219968 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d02a47-94b0-4e4e-a574-f0b202d34a57 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:52.220570 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d02a47-94b0-4e4e-a574-f0b202d34a57 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:52.223424 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d02a47-94b0-4e4e-a574-f0b202d34a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:52.251526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ee16b-7ddc-4f3a-a948-7599d0658c1a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:52.276481 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-63d02a47-94b0-4e4e-a574-f0b202d34a57 None None] MFA Rules not processed for user `794a266461694f8bacc61ba4ce613ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:52.281201 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-63d02a47-94b0-4e4e-a574-f0b202d34a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:52.295657 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:52.409546 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51702bce-de31-4869-b8c9-251ff034a44a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33a0844a1c8f4ea4932e12d53e5ee9ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:52.410431 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51702bce-de31-4869-b8c9-251ff034a44a service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:52.410784 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51702bce-de31-4869-b8c9-251ff034a44a service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:52.411164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51702bce-de31-4869-b8c9-251ff034a44a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:52.413407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-51702bce-de31-4869-b8c9-251ff034a44a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:52.432436 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51702bce-de31-4869-b8c9-251ff034a44a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:52.434391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51702bce-de31-4869-b8c9-251ff034a44a service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:52.435321 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1068/2144] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:52 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:52.448590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d02a47-94b0-4e4e-a574-f0b202d34a57 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:52.450363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-63d02a47-94b0-4e4e-a574-f0b202d34a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:52.454545 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1077/2145] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:52.514864 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:52.517857 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51702bce-de31-4869-b8c9-251ff034a44a service placement] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:52.520815 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-51702bce-de31-4869-b8c9-251ff034a44a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:52.552332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d02a47-94b0-4e4e-a574-f0b202d34a57 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:52.558093 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15d416-b655-4532-a5aa-02c668703985 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:52.559249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15d416-b655-4532-a5aa-02c668703985 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:52.559249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15d416-b655-4532-a5aa-02c668703985 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:52.559616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15d416-b655-4532-a5aa-02c668703985 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:52.571790 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a15d416-b655-4532-a5aa-02c668703985 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:52.576644 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a15d416-b655-4532-a5aa-02c668703985 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:52.607564 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1069/2146] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:52 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:52.637953 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51702bce-de31-4869-b8c9-251ff034a44a service placement] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:52.654264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f2ecd-f4ce-4f45-b9b3-fb0d4bb4bfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:52.655037 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f2ecd-f4ce-4f45-b9b3-fb0d4bb4bfc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:52.655730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f2ecd-f4ce-4f45-b9b3-fb0d4bb4bfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:52.656016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f2ecd-f4ce-4f45-b9b3-fb0d4bb4bfc5 admin admin] PATH_INFO: `/v3/projects/61de8fb05c3a45de90ff3238b61452bb` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:52.659102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f2ecd-f4ce-4f45-b9b3-fb0d4bb4bfc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61de8fb05c3a45de90ff3238b61452bb)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:52.664624 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f2ecd-f4ce-4f45-b9b3-fb0d4bb4bfc5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:52.750961 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f16f2ecd-f4ce-4f45-b9b3-fb0d4bb4bfc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61de8fb05c3a45de90ff3238b61452bb', 'request_id': 'req-f16f2ecd-f4ce-4f45-b9b3-fb0d4bb4bfc5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:52.757783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f16f2ecd-f4ce-4f45-b9b3-fb0d4bb4bfc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61de8fb05c3a45de90ff3238b61452bb', 'request_id': 'req-f16f2ecd-f4ce-4f45-b9b3-fb0d4bb4bfc5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:52.799183 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f16f2ecd-f4ce-4f45-b9b3-fb0d4bb4bfc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61de8fb05c3a45de90ff3238b61452bb', 'request_id': 'req-f16f2ecd-f4ce-4f45-b9b3-fb0d4bb4bfc5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:52.827843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f16f2ecd-f4ce-4f45-b9b3-fb0d4bb4bfc5 admin admin] The token cache is being invalidate because project 61de8fb05c3a45de90ff3238b61452bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:58:52.827991 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f16f2ecd-f4ce-4f45-b9b3-fb0d4bb4bfc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:52.828823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f16f2ecd-f4ce-4f45-b9b3-fb0d4bb4bfc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:58:52.834536 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1078/2147] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:58:52 2022] DELETE /identity/v3/projects/61de8fb05c3a45de90ff3238b61452bb => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:52.836225 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204dcc6c-01ed-4208-9da3-55549ee16cee None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:52.836669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204dcc6c-01ed-4208-9da3-55549ee16cee None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:52.836669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204dcc6c-01ed-4208-9da3-55549ee16cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:52.860608 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-204dcc6c-01ed-4208-9da3-55549ee16cee None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:52.862021 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-204dcc6c-01ed-4208-9da3-55549ee16cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:52.864592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f2ecd-f4ce-4f45-b9b3-fb0d4bb4bfc5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:52.866496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f16f2ecd-f4ce-4f45-b9b3-fb0d4bb4bfc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:52.953457 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:52.975141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f2ecd-f4ce-4f45-b9b3-fb0d4bb4bfc5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:52.980102 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f83ec92-7f0e-4b98-84dd-f9bcc0477701 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:52.980345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1070/2148] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:52.981090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83ec92-7f0e-4b98-84dd-f9bcc0477701 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:52.981420 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83ec92-7f0e-4b98-84dd-f9bcc0477701 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:52.981875 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83ec92-7f0e-4b98-84dd-f9bcc0477701 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:52.987621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2f83ec92-7f0e-4b98-84dd-f9bcc0477701 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:52.987979 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cd958-3830-43ef-93c7-823473825122 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:52.988375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cd958-3830-43ef-93c7-823473825122 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:52.988762 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cd958-3830-43ef-93c7-823473825122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:53.013756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-c89cd958-3830-43ef-93c7-823473825122 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:53.015647 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c89cd958-3830-43ef-93c7-823473825122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:53.044216 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1071/2149] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:58:53.048064 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cd958-3830-43ef-93c7-823473825122 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:53.060999 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cd958-3830-43ef-93c7-823473825122 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:53.066115 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef85aca-8421-4baa-b8f2-4b88b6447a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:53.066885 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef85aca-8421-4baa-b8f2-4b88b6447a2f admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:53.067318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef85aca-8421-4baa-b8f2-4b88b6447a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:53.067634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef85aca-8421-4baa-b8f2-4b88b6447a2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:53.068406 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef85aca-8421-4baa-b8f2-4b88b6447a2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:58:53.069053 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef85aca-8421-4baa-b8f2-4b88b6447a2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:53.070710 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef85aca-8421-4baa-b8f2-4b88b6447a2f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:53.093873 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1072/2150] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:58:53.099871 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef85aca-8421-4baa-b8f2-4b88b6447a2f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:53.109167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f83ec92-7f0e-4b98-84dd-f9bcc0477701 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:53.110331 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f83ec92-7f0e-4b98-84dd-f9bcc0477701 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:53.118461 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef85aca-8421-4baa-b8f2-4b88b6447a2f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:53.125019 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1079/2151] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:52 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:53.146256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbf3e0-48ce-48d8-8582-8d0668e813e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:53.147330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbf3e0-48ce-48d8-8582-8d0668e813e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:53.147840 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbf3e0-48ce-48d8-8582-8d0668e813e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:53.148221 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbf3e0-48ce-48d8-8582-8d0668e813e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:53.149094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbbf3e0-48ce-48d8-8582-8d0668e813e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:53.154601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbbf3e0-48ce-48d8-8582-8d0668e813e3 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:53.205634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1073/2152] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:53 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:53.210578 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f83ec92-7f0e-4b98-84dd-f9bcc0477701 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:53.227728 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f83ec92-7f0e-4b98-84dd-f9bcc0477701 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:53.311000 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289129b-ecfb-44c3-b9af-cc42a00ecb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:53.311628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289129b-ecfb-44c3-b9af-cc42a00ecb2e admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:53.311977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289129b-ecfb-44c3-b9af-cc42a00ecb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:53.312506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289129b-ecfb-44c3-b9af-cc42a00ecb2e admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:53.313243 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289129b-ecfb-44c3-b9af-cc42a00ecb2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:53.315360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289129b-ecfb-44c3-b9af-cc42a00ecb2e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:53.338895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1080/2153] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:53 2022] POST /identity/v3/users => generated 283 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:53.343047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbf3e0-48ce-48d8-8582-8d0668e813e3 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:53.353941 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbf3e0-48ce-48d8-8582-8d0668e813e3 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:53.365455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836b144-02b5-41ab-b862-27572e6b2ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:53.366473 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836b144-02b5-41ab-b862-27572e6b2ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:53.366916 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836b144-02b5-41ab-b862-27572e6b2ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:53.367192 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836b144-02b5-41ab-b862-27572e6b2ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:53.367897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836b144-02b5-41ab-b862-27572e6b2ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:53.370088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836b144-02b5-41ab-b862-27572e6b2ef6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:53.397594 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1074/2154] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:53.401682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289129b-ecfb-44c3-b9af-cc42a00ecb2e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:53.414656 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289129b-ecfb-44c3-b9af-cc42a00ecb2e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:53.421791 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1169d-7225-46c3-9547-3d438ae794d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:53.422477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1169d-7225-46c3-9547-3d438ae794d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:53.422805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1169d-7225-46c3-9547-3d438ae794d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:53.423086 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1169d-7225-46c3-9547-3d438ae794d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:53.423839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1169d-7225-46c3-9547-3d438ae794d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:53.425563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1169d-7225-46c3-9547-3d438ae794d1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:53.446173 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1081/2155] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:58:53.450636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836b144-02b5-41ab-b862-27572e6b2ef6 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:53.461162 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836b144-02b5-41ab-b862-27572e6b2ef6 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:53.467245 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58263b-0090-42fe-9c5a-091f677b07a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:53.467960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58263b-0090-42fe-9c5a-091f677b07a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:53.468297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58263b-0090-42fe-9c5a-091f677b07a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:53.468550 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58263b-0090-42fe-9c5a-091f677b07a2 admin admin] PATH_INFO: `/v3/projects/ae769ec5950844b5940fbc977f50ca5b/users/f701cf08283b4ee78b9eaa502890507f/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:53.486373 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58263b-0090-42fe-9c5a-091f677b07a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae769ec5950844b5940fbc977f50ca5b, user_id=f701cf08283b4ee78b9eaa502890507f, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:53.489724 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58263b-0090-42fe-9c5a-091f677b07a2 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:53.516873 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1075/2156] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:53 2022] PUT /identity/v3/projects/ae769ec5950844b5940fbc977f50ca5b/users/f701cf08283b4ee78b9eaa502890507f/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:58:53.529148 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804559a-2e30-4141-9806-f26410abc694 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:53.529342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804559a-2e30-4141-9806-f26410abc694 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:53.529609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804559a-2e30-4141-9806-f26410abc694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:53.584716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1804559a-2e30-4141-9806-f26410abc694 None None] MFA Rules not processed for user `f701cf08283b4ee78b9eaa502890507f`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:53.585949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1804559a-2e30-4141-9806-f26410abc694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:53.680249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1082/2157] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:53.694170 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58263b-0090-42fe-9c5a-091f677b07a2 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:53.787202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58263b-0090-42fe-9c5a-091f677b07a2 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:53.794958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c2928-cbff-4d91-97cb-f20cb9154e23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:53.795801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c2928-cbff-4d91-97cb-f20cb9154e23 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:53.796541 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c2928-cbff-4d91-97cb-f20cb9154e23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:53.796823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c2928-cbff-4d91-97cb-f20cb9154e23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:53.807795 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c2928-cbff-4d91-97cb-f20cb9154e23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:53.809492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c2928-cbff-4d91-97cb-f20cb9154e23 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:53.813732 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:53.821270 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1076/2158] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:58:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:58:56.314424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1a760-993b-438d-be33-9dcc5202f1f0 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:56.314424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1a760-993b-438d-be33-9dcc5202f1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:56.314424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1a760-993b-438d-be33-9dcc5202f1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:56.323112 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:58:56.346818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-27d1a760-993b-438d-be33-9dcc5202f1f0 None None] MFA Rules not processed for user `46b3aa99a5e84ef2a8df193628feb0d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:56.352109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-27d1a760-993b-438d-be33-9dcc5202f1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:56.440932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1083/2159] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:58:56.448311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c2928-cbff-4d91-97cb-f20cb9154e23 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:56.449947 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-486c2928-cbff-4d91-97cb-f20cb9154e23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:56.547812 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c2928-cbff-4d91-97cb-f20cb9154e23 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:56.554658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938897f-5883-4b8d-9201-e468062848c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:56.555487 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938897f-5883-4b8d-9201-e468062848c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:56.556088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938897f-5883-4b8d-9201-e468062848c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:56.557255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938897f-5883-4b8d-9201-e468062848c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:56.558329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938897f-5883-4b8d-9201-e468062848c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:56.560455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938897f-5883-4b8d-9201-e468062848c8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:56.597629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1077/2160] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:58:56 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:56.598988 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1a760-993b-438d-be33-9dcc5202f1f0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:56.614503 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1a760-993b-438d-be33-9dcc5202f1f0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:56.686272 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ce57d-8523-42dc-ba31-fe8331fefc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:56.686961 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ce57d-8523-42dc-ba31-fe8331fefc26 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:56.687246 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ce57d-8523-42dc-ba31-fe8331fefc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:56.687518 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ce57d-8523-42dc-ba31-fe8331fefc26 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:56.688210 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ce57d-8523-42dc-ba31-fe8331fefc26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:56.690179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ce57d-8523-42dc-ba31-fe8331fefc26 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:56.718246 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1084/2161] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:58:56 2022] POST /identity/v3/users => generated 296 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:58:56.722109 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938897f-5883-4b8d-9201-e468062848c8 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:56.733824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938897f-5883-4b8d-9201-e468062848c8 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:56.738638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d61409-3975-49a1-bd61-de3519d0b08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:56.739262 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d61409-3975-49a1-bd61-de3519d0b08f admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:56.739614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d61409-3975-49a1-bd61-de3519d0b08f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:56.739842 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d61409-3975-49a1-bd61-de3519d0b08f admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:56.740539 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d61409-3975-49a1-bd61-de3519d0b08f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:56.742148 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d61409-3975-49a1-bd61-de3519d0b08f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:56.757123 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1078/2162] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:56.758094 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ce57d-8523-42dc-ba31-fe8331fefc26 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:56.771870 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ce57d-8523-42dc-ba31-fe8331fefc26 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:56.777911 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47458790-5c33-45fa-9120-8e714aa4132c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:56.778853 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47458790-5c33-45fa-9120-8e714aa4132c admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:56.779238 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47458790-5c33-45fa-9120-8e714aa4132c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:56.779678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47458790-5c33-45fa-9120-8e714aa4132c admin admin] PATH_INFO: `/v3/projects/364a8d179c244dcd83327a69ec26c5e0/users/fd47057b9c334e1eb36ff49bd3273171/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:56.801767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47458790-5c33-45fa-9120-8e714aa4132c admin admin] RBAC: Authorizing `identity:create_grant(project_id=364a8d179c244dcd83327a69ec26c5e0, user_id=fd47057b9c334e1eb36ff49bd3273171, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:56.804539 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47458790-5c33-45fa-9120-8e714aa4132c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:56.835597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1085/2163] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:56 2022] PUT /identity/v3/projects/364a8d179c244dcd83327a69ec26c5e0/users/fd47057b9c334e1eb36ff49bd3273171/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:56.840971 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d61409-3975-49a1-bd61-de3519d0b08f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:56.854517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d61409-3975-49a1-bd61-de3519d0b08f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:56.931460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf020cb-7afe-405f-b281-423de5c6e0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:56.932121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf020cb-7afe-405f-b281-423de5c6e0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:56.932409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf020cb-7afe-405f-b281-423de5c6e0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:56.932685 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf020cb-7afe-405f-b281-423de5c6e0a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:56.933513 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf020cb-7afe-405f-b281-423de5c6e0a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:56.935477 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf020cb-7afe-405f-b281-423de5c6e0a9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:56.949951 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1079/2164] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:58:56.953888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47458790-5c33-45fa-9120-8e714aa4132c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:58:56.965954 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47458790-5c33-45fa-9120-8e714aa4132c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:58:56.971413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce2eeb-2eaa-4c3b-ba34-0b7c6780a026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:58:56.972176 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce2eeb-2eaa-4c3b-ba34-0b7c6780a026 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:56.972935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce2eeb-2eaa-4c3b-ba34-0b7c6780a026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:56.972935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce2eeb-2eaa-4c3b-ba34-0b7c6780a026 admin admin] PATH_INFO: `/v3/projects/364a8d179c244dcd83327a69ec26c5e0/users/fd47057b9c334e1eb36ff49bd3273171/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:56.975095 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce2eeb-2eaa-4c3b-ba34-0b7c6780a026 admin admin] RBAC: Authorizing `identity:create_grant(project_id=364a8d179c244dcd83327a69ec26c5e0, user_id=fd47057b9c334e1eb36ff49bd3273171, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:58:56.979634 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce2eeb-2eaa-4c3b-ba34-0b7c6780a026 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:58:56.993070 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1086/2165] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:58:56 2022] PUT /identity/v3/projects/364a8d179c244dcd83327a69ec26c5e0/users/fd47057b9c334e1eb36ff49bd3273171/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:58:58.317075 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a68683-e62c-4773-8731-dea9f7c9f680 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:58:58.317585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a68683-e62c-4773-8731-dea9f7c9f680 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:58:58.318612 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a68683-e62c-4773-8731-dea9f7c9f680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:58:58.343453 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-12a68683-e62c-4773-8731-dea9f7c9f680 None None] MFA Rules not processed for user `8ee5364270094596b64b08a8ac56f6ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:58:58.345837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-12a68683-e62c-4773-8731-dea9f7c9f680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:58:58.435555 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1080/2166] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:58:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:59:01.699140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce2eeb-2eaa-4c3b-ba34-0b7c6780a026 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:01.701106 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-80ce2eeb-2eaa-4c3b-ba34-0b7c6780a026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:01.785447 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:01.819217 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce2eeb-2eaa-4c3b-ba34-0b7c6780a026 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:01.826007 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d75c9-423e-4d17-a0e4-a7d366d2bcfd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33a0844a1c8f4ea4932e12d53e5ee9ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:01.826804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d75c9-423e-4d17-a0e4-a7d366d2bcfd service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:01.827265 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d75c9-423e-4d17-a0e4-a7d366d2bcfd service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:01.827918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d75c9-423e-4d17-a0e4-a7d366d2bcfd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:01.830114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-578d75c9-423e-4d17-a0e4-a7d366d2bcfd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:01.845354 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d75c9-423e-4d17-a0e4-a7d366d2bcfd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:01.847919 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d75c9-423e-4d17-a0e4-a7d366d2bcfd service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:01.862619 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1087/2167] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:59:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:59:03.346477 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e1a86-8361-4fd6-9e4d-f3601e3d5414 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:03.347380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e1a86-8361-4fd6-9e4d-f3601e3d5414 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:03.347882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e1a86-8361-4fd6-9e4d-f3601e3d5414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:03.380884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:03.384081 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-978e1a86-8361-4fd6-9e4d-f3601e3d5414 None None] MFA Rules not processed for user `fd47057b9c334e1eb36ff49bd3273171`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:03.385969 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-978e1a86-8361-4fd6-9e4d-f3601e3d5414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:03.518954 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1081/2168] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:59:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:03.533195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d75c9-423e-4d17-a0e4-a7d366d2bcfd service placement] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:03.534754 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-578d75c9-423e-4d17-a0e4-a7d366d2bcfd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:03.605241 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e1a86-8361-4fd6-9e4d-f3601e3d5414 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:03.606919 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-978e1a86-8361-4fd6-9e4d-f3601e3d5414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:03.640449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d75c9-423e-4d17-a0e4-a7d366d2bcfd service placement] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:03.647237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466598c-dc5d-4193-a290-3dba5e30e950 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:03.647930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466598c-dc5d-4193-a290-3dba5e30e950 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:03.648317 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466598c-dc5d-4193-a290-3dba5e30e950 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:03.649078 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466598c-dc5d-4193-a290-3dba5e30e950 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:03.703344 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e1a86-8361-4fd6-9e4d-f3601e3d5414 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:03.714114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b705815-1e96-4a17-af7f-9de03604f188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:03.714938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b705815-1e96-4a17-af7f-9de03604f188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:03.720270 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b705815-1e96-4a17-af7f-9de03604f188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:03.720270 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b705815-1e96-4a17-af7f-9de03604f188 admin admin] PATH_INFO: `/v3/users/f97a11029ba543c286048f3f6e934324` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:03.720270 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b705815-1e96-4a17-af7f-9de03604f188 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f97a11029ba543c286048f3f6e934324)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:03.724389 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b705815-1e96-4a17-af7f-9de03604f188 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:03.730510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466598c-dc5d-4193-a290-3dba5e30e950 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:03.733448 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466598c-dc5d-4193-a290-3dba5e30e950 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:03.756800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1088/2169] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:59:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:59:03.809257 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3b705815-1e96-4a17-af7f-9de03604f188 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f97a11029ba543c286048f3f6e934324'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:03.819079 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3b705815-1e96-4a17-af7f-9de03604f188 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f97a11029ba543c286048f3f6e934324'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:03.839851 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3b705815-1e96-4a17-af7f-9de03604f188 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f97a11029ba543c286048f3f6e934324'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:03.849152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3b705815-1e96-4a17-af7f-9de03604f188 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f97a11029ba543c286048f3f6e934324'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:03.849913 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3b705815-1e96-4a17-af7f-9de03604f188 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f97a11029ba543c286048f3f6e934324'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:03.866990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1082/2170] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:59:03 2022] DELETE /identity/v3/users/f97a11029ba543c286048f3f6e934324 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:59:03.869962 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9ef1b-8627-4e94-8433-a7a86661d711 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:03.870413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9ef1b-8627-4e94-8433-a7a86661d711 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:03.870825 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9ef1b-8627-4e94-8433-a7a86661d711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:03.940804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e4d9ef1b-8627-4e94-8433-a7a86661d711 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:03.943849 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e4d9ef1b-8627-4e94-8433-a7a86661d711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:04.045755 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1089/2171] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:04.061788 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b705815-1e96-4a17-af7f-9de03604f188 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:04.063311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3b705815-1e96-4a17-af7f-9de03604f188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:04.090032 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:04.171176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b705815-1e96-4a17-af7f-9de03604f188 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:04.183367 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde93c9-3f21-4cde-bcf6-f72363999ea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:04.184022 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde93c9-3f21-4cde-bcf6-f72363999ea6 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:04.184373 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde93c9-3f21-4cde-bcf6-f72363999ea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:04.184649 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde93c9-3f21-4cde-bcf6-f72363999ea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:04.212116 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde93c9-3f21-4cde-bcf6-f72363999ea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:04.214605 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde93c9-3f21-4cde-bcf6-f72363999ea6 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:04.252190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1083/2172] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:59:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:59:05.351725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9ef1b-8627-4e94-8433-a7a86661d711 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:05.353470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e4d9ef1b-8627-4e94-8433-a7a86661d711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:05.406932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9ef1b-8627-4e94-8433-a7a86661d711 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:05.430412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9db9bb-1bcb-4245-ad6a-19ec63844942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:05.432799 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9db9bb-1bcb-4245-ad6a-19ec63844942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:05.434193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9db9bb-1bcb-4245-ad6a-19ec63844942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:05.434803 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9db9bb-1bcb-4245-ad6a-19ec63844942 admin admin] PATH_INFO: `/v3/projects/5d08444163c544d58245c44f97d74f4a` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:05.440280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9db9bb-1bcb-4245-ad6a-19ec63844942 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d08444163c544d58245c44f97d74f4a)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:05.450749 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9db9bb-1bcb-4245-ad6a-19ec63844942 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:05.483564 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:05.569880 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3b9db9bb-1bcb-4245-ad6a-19ec63844942 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d08444163c544d58245c44f97d74f4a', 'request_id': 'req-3b9db9bb-1bcb-4245-ad6a-19ec63844942'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:05.579685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3b9db9bb-1bcb-4245-ad6a-19ec63844942 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d08444163c544d58245c44f97d74f4a', 'request_id': 'req-3b9db9bb-1bcb-4245-ad6a-19ec63844942'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:05.592430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3b9db9bb-1bcb-4245-ad6a-19ec63844942 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d08444163c544d58245c44f97d74f4a', 'request_id': 'req-3b9db9bb-1bcb-4245-ad6a-19ec63844942'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:05.616042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3b9db9bb-1bcb-4245-ad6a-19ec63844942 admin admin] The token cache is being invalidate because project 5d08444163c544d58245c44f97d74f4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:59:05.616718 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3b9db9bb-1bcb-4245-ad6a-19ec63844942 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:05.618312 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3b9db9bb-1bcb-4245-ad6a-19ec63844942 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:05.620303 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1090/2173] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:59:05 2022] DELETE /identity/v3/projects/5d08444163c544d58245c44f97d74f4a => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:59:05.624817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde93c9-3f21-4cde-bcf6-f72363999ea6 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:05.640047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde93c9-3f21-4cde-bcf6-f72363999ea6 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:05.757438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adecaf1-648c-4b27-8310-7f90a55cd506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:05.758767 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adecaf1-648c-4b27-8310-7f90a55cd506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:05.759586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adecaf1-648c-4b27-8310-7f90a55cd506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:05.759939 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adecaf1-648c-4b27-8310-7f90a55cd506 admin admin] PATH_INFO: `/v3/users/794a266461694f8bacc61ba4ce613ec9` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:05.762259 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adecaf1-648c-4b27-8310-7f90a55cd506 admin admin] RBAC: Authorizing `identity:delete_user(user_id=794a266461694f8bacc61ba4ce613ec9)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:05.765990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adecaf1-648c-4b27-8310-7f90a55cd506 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:05.853114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4adecaf1-648c-4b27-8310-7f90a55cd506 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '794a266461694f8bacc61ba4ce613ec9'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:05.867286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4adecaf1-648c-4b27-8310-7f90a55cd506 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '794a266461694f8bacc61ba4ce613ec9'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:05.890686 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4adecaf1-648c-4b27-8310-7f90a55cd506 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '794a266461694f8bacc61ba4ce613ec9'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:05.902738 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4adecaf1-648c-4b27-8310-7f90a55cd506 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '794a266461694f8bacc61ba4ce613ec9'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:05.903853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4adecaf1-648c-4b27-8310-7f90a55cd506 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '794a266461694f8bacc61ba4ce613ec9'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:05.915026 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1084/2174] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:59:05 2022] DELETE /identity/v3/users/794a266461694f8bacc61ba4ce613ec9 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:59:06.264428 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9db9bb-1bcb-4245-ad6a-19ec63844942 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:06.266306 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3b9db9bb-1bcb-4245-ad6a-19ec63844942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:06.300395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9db9bb-1bcb-4245-ad6a-19ec63844942 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:06.378844 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e5125e-106f-42d4-9614-0e9b8260c9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:06.379497 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5125e-106f-42d4-9614-0e9b8260c9bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:06.379779 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5125e-106f-42d4-9614-0e9b8260c9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:06.380048 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5125e-106f-42d4-9614-0e9b8260c9bd admin admin] PATH_INFO: `/v3/projects/b0f39107b8594d97b64121b5b2a011f2` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:06.381333 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e5125e-106f-42d4-9614-0e9b8260c9bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f39107b8594d97b64121b5b2a011f2)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:06.384239 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e5125e-106f-42d4-9614-0e9b8260c9bd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:06.444389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-90e5125e-106f-42d4-9614-0e9b8260c9bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f39107b8594d97b64121b5b2a011f2', 'request_id': 'req-90e5125e-106f-42d4-9614-0e9b8260c9bd'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:06.451929 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-90e5125e-106f-42d4-9614-0e9b8260c9bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f39107b8594d97b64121b5b2a011f2', 'request_id': 'req-90e5125e-106f-42d4-9614-0e9b8260c9bd'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:06.459863 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-90e5125e-106f-42d4-9614-0e9b8260c9bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f39107b8594d97b64121b5b2a011f2', 'request_id': 'req-90e5125e-106f-42d4-9614-0e9b8260c9bd'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:06.472461 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-90e5125e-106f-42d4-9614-0e9b8260c9bd admin admin] The token cache is being invalidate because project b0f39107b8594d97b64121b5b2a011f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:59:06.472748 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-90e5125e-106f-42d4-9614-0e9b8260c9bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:06.473383 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-90e5125e-106f-42d4-9614-0e9b8260c9bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:06.474866 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1091/2175] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:59:06 2022] DELETE /identity/v3/projects/b0f39107b8594d97b64121b5b2a011f2 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:59:06.479822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a3901-e4b7-4688-b699-fad9eecd7a39 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:06.480123 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a3901-e4b7-4688-b699-fad9eecd7a39 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:06.480439 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a3901-e4b7-4688-b699-fad9eecd7a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:06.500761 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d27a3901-e4b7-4688-b699-fad9eecd7a39 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:06.502164 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d27a3901-e4b7-4688-b699-fad9eecd7a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:06.540978 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:06.570990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1085/2176] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:59:06.580007 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38e44b-4a9c-4983-80fc-2384e1a6a8de None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:06.580415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38e44b-4a9c-4983-80fc-2384e1a6a8de None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:06.580698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38e44b-4a9c-4983-80fc-2384e1a6a8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:06.602920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-be38e44b-4a9c-4983-80fc-2384e1a6a8de None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:06.604388 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-be38e44b-4a9c-4983-80fc-2384e1a6a8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:06.613328 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1092/2177] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:06.617881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a3901-e4b7-4688-b699-fad9eecd7a39 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:06.629225 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a3901-e4b7-4688-b699-fad9eecd7a39 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:06.634148 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02373e6-c3ea-4a9c-bfec-72ac16067fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:06.634779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02373e6-c3ea-4a9c-bfec-72ac16067fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:06.635617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02373e6-c3ea-4a9c-bfec-72ac16067fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:06.635911 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02373e6-c3ea-4a9c-bfec-72ac16067fb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:06.636693 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02373e6-c3ea-4a9c-bfec-72ac16067fb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:59:06.636998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02373e6-c3ea-4a9c-bfec-72ac16067fb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:06.638605 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02373e6-c3ea-4a9c-bfec-72ac16067fb1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:06.652008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1086/2178] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:59:06.656257 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38e44b-4a9c-4983-80fc-2384e1a6a8de None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:06.666496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38e44b-4a9c-4983-80fc-2384e1a6a8de None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:06.671310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bb97e-5ffe-4aeb-99ac-7612044ad41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:06.672069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bb97e-5ffe-4aeb-99ac-7612044ad41a admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:06.672426 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bb97e-5ffe-4aeb-99ac-7612044ad41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:06.672696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bb97e-5ffe-4aeb-99ac-7612044ad41a admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:06.673328 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9bb97e-5ffe-4aeb-99ac-7612044ad41a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:06.675253 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9bb97e-5ffe-4aeb-99ac-7612044ad41a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:06.699302 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:06.709583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1093/2179] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:59:06 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:59:06.714323 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02373e6-c3ea-4a9c-bfec-72ac16067fb1 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:06.727265 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02373e6-c3ea-4a9c-bfec-72ac16067fb1 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:06.798678 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113481be-db48-459f-8a4c-1a078c700649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:06.799304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113481be-db48-459f-8a4c-1a078c700649 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:06.799588 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113481be-db48-459f-8a4c-1a078c700649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:06.799864 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113481be-db48-459f-8a4c-1a078c700649 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:06.800588 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113481be-db48-459f-8a4c-1a078c700649 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:06.802574 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113481be-db48-459f-8a4c-1a078c700649 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:06.836450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1087/2180] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:59:06 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:59:06.839659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bb97e-5ffe-4aeb-99ac-7612044ad41a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:06.851286 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bb97e-5ffe-4aeb-99ac-7612044ad41a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:06.864427 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb26499-8188-42df-ad59-8cf3769e1a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:06.865247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb26499-8188-42df-ad59-8cf3769e1a2f admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:06.865339 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb26499-8188-42df-ad59-8cf3769e1a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:06.865941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb26499-8188-42df-ad59-8cf3769e1a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:06.866604 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb26499-8188-42df-ad59-8cf3769e1a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:06.868222 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb26499-8188-42df-ad59-8cf3769e1a2f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:06.880144 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1094/2181] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:59:06.883961 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113481be-db48-459f-8a4c-1a078c700649 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:06.894824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113481be-db48-459f-8a4c-1a078c700649 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:06.899813 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cad8bd-e5d0-4a9b-982c-9e84a50cf48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:06.901103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cad8bd-e5d0-4a9b-982c-9e84a50cf48a admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:06.901388 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cad8bd-e5d0-4a9b-982c-9e84a50cf48a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:06.902105 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cad8bd-e5d0-4a9b-982c-9e84a50cf48a admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:06.902861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cad8bd-e5d0-4a9b-982c-9e84a50cf48a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:06.904516 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cad8bd-e5d0-4a9b-982c-9e84a50cf48a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:06.918262 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1088/2182] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:59:06.922285 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb26499-8188-42df-ad59-8cf3769e1a2f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:06.952956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb26499-8188-42df-ad59-8cf3769e1a2f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:06.959688 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510ec26-eeca-4760-a798-42cd8f4f5f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:06.960683 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510ec26-eeca-4760-a798-42cd8f4f5f8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:06.961201 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510ec26-eeca-4760-a798-42cd8f4f5f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:06.961658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510ec26-eeca-4760-a798-42cd8f4f5f8b admin admin] PATH_INFO: `/v3/projects/d5382ca4bd054fec89a6532a732aa72e/users/dcc31a9543cf459b87b42914c1f6fb44/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:06.996010 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510ec26-eeca-4760-a798-42cd8f4f5f8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5382ca4bd054fec89a6532a732aa72e, user_id=dcc31a9543cf459b87b42914c1f6fb44, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:06.999310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510ec26-eeca-4760-a798-42cd8f4f5f8b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:07.020059 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1095/2183] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:59:06 2022] PUT /identity/v3/projects/d5382ca4bd054fec89a6532a732aa72e/users/dcc31a9543cf459b87b42914c1f6fb44/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:59:07.031250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca67a82-e804-48a4-bba3-cd9660d0f986 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:07.031700 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca67a82-e804-48a4-bba3-cd9660d0f986 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:07.032007 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca67a82-e804-48a4-bba3-cd9660d0f986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:07.083583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-eca67a82-e804-48a4-bba3-cd9660d0f986 None None] MFA Rules not processed for user `dcc31a9543cf459b87b42914c1f6fb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:07.085071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-eca67a82-e804-48a4-bba3-cd9660d0f986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:07.208140 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1089/2184] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:59:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:59:07.210765 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510ec26-eeca-4760-a798-42cd8f4f5f8b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:07.299622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510ec26-eeca-4760-a798-42cd8f4f5f8b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:07.305252 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdde33-aa53-4111-9852-71808558daaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:07.305943 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdde33-aa53-4111-9852-71808558daaf admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:07.306426 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdde33-aa53-4111-9852-71808558daaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:07.306765 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdde33-aa53-4111-9852-71808558daaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:07.308485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdde33-aa53-4111-9852-71808558daaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:07.310523 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdde33-aa53-4111-9852-71808558daaf admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:07.349320 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1096/2185] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:59:07 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:59:07.349950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca67a82-e804-48a4-bba3-cd9660d0f986 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:07.362715 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca67a82-e804-48a4-bba3-cd9660d0f986 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:07.435546 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:07.493039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381aafe-641b-4355-8733-a765901f9a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:07.493039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381aafe-641b-4355-8733-a765901f9a58 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:07.493039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381aafe-641b-4355-8733-a765901f9a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:07.493417 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381aafe-641b-4355-8733-a765901f9a58 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:07.493950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381aafe-641b-4355-8733-a765901f9a58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:07.496815 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381aafe-641b-4355-8733-a765901f9a58 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:07.522328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1090/2186] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:59:07 2022] POST /identity/v3/users => generated 299 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:59:07.526252 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdde33-aa53-4111-9852-71808558daaf admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:07.538084 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdde33-aa53-4111-9852-71808558daaf admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:07.545031 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ab5c1-c186-48e0-8539-d8a21008fac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:07.545975 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ab5c1-c186-48e0-8539-d8a21008fac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:07.546416 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ab5c1-c186-48e0-8539-d8a21008fac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:07.546845 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ab5c1-c186-48e0-8539-d8a21008fac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:07.547691 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ab5c1-c186-48e0-8539-d8a21008fac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:07.549606 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ab5c1-c186-48e0-8539-d8a21008fac9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:07.566584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1097/2187] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:59:07.571039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381aafe-641b-4355-8733-a765901f9a58 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:07.583695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381aafe-641b-4355-8733-a765901f9a58 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:07.597385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f472fc-dd2b-4a8d-bd5c-cebe6110dd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:07.598066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f472fc-dd2b-4a8d-bd5c-cebe6110dd1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:07.598342 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f472fc-dd2b-4a8d-bd5c-cebe6110dd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:07.598612 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f472fc-dd2b-4a8d-bd5c-cebe6110dd1f admin admin] PATH_INFO: `/v3/projects/d05d11a2732b47feaf4210ab47b9fd60/users/df9b20d3c97b4257a8c99398efdf5b6d/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:07.616213 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f472fc-dd2b-4a8d-bd5c-cebe6110dd1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d05d11a2732b47feaf4210ab47b9fd60, user_id=df9b20d3c97b4257a8c99398efdf5b6d, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:07.618999 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f472fc-dd2b-4a8d-bd5c-cebe6110dd1f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:07.634065 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1091/2188] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:59:07 2022] PUT /identity/v3/projects/d05d11a2732b47feaf4210ab47b9fd60/users/df9b20d3c97b4257a8c99398efdf5b6d/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:59:07.638764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ab5c1-c186-48e0-8539-d8a21008fac9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:07.648760 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ab5c1-c186-48e0-8539-d8a21008fac9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:07.721246 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d699d-fb6e-4283-a545-cb05091bd364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:07.721246 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d699d-fb6e-4283-a545-cb05091bd364 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:07.721246 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d699d-fb6e-4283-a545-cb05091bd364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:07.721246 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d699d-fb6e-4283-a545-cb05091bd364 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:07.721702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d699d-fb6e-4283-a545-cb05091bd364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:07.722193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d699d-fb6e-4283-a545-cb05091bd364 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:07.738412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1098/2189] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:59:07.739717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f472fc-dd2b-4a8d-bd5c-cebe6110dd1f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:07.754374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f472fc-dd2b-4a8d-bd5c-cebe6110dd1f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:07.760720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a9f49-a84f-454f-b4a0-9c62136d8fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:07.760720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a9f49-a84f-454f-b4a0-9c62136d8fa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:07.761198 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a9f49-a84f-454f-b4a0-9c62136d8fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:07.761624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a9f49-a84f-454f-b4a0-9c62136d8fa4 admin admin] PATH_INFO: `/v3/projects/d05d11a2732b47feaf4210ab47b9fd60/users/df9b20d3c97b4257a8c99398efdf5b6d/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:07.764515 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a9f49-a84f-454f-b4a0-9c62136d8fa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d05d11a2732b47feaf4210ab47b9fd60, user_id=df9b20d3c97b4257a8c99398efdf5b6d, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:07.767192 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a9f49-a84f-454f-b4a0-9c62136d8fa4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:07.780402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1092/2190] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:59:07 2022] PUT /identity/v3/projects/d05d11a2732b47feaf4210ab47b9fd60/users/df9b20d3c97b4257a8c99398efdf5b6d/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:59:07.792911 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff458c-cea7-45c7-b145-114e3fb27670 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:07.793218 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff458c-cea7-45c7-b145-114e3fb27670 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:07.793497 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff458c-cea7-45c7-b145-114e3fb27670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:07.830656 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e9ff458c-cea7-45c7-b145-114e3fb27670 None None] MFA Rules not processed for user `df9b20d3c97b4257a8c99398efdf5b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:07.831914 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e9ff458c-cea7-45c7-b145-114e3fb27670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:07.863437 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:07.923765 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1099/2191] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:59:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:07.936534 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a9f49-a84f-454f-b4a0-9c62136d8fa4 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:07.940420 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-343a9f49-a84f-454f-b4a0-9c62136d8fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:08.022292 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a9f49-a84f-454f-b4a0-9c62136d8fa4 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:08.027344 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6320b71-49cd-4b4e-acd3-d801990f616e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:08.027942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6320b71-49cd-4b4e-acd3-d801990f616e service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:08.028334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6320b71-49cd-4b4e-acd3-d801990f616e service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:08.028591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6320b71-49cd-4b4e-acd3-d801990f616e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:08.108595 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6320b71-49cd-4b4e-acd3-d801990f616e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:08.110320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6320b71-49cd-4b4e-acd3-d801990f616e service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:08.120792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1093/2192] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:59:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:59:17.920815 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=19 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:18.116136 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=76,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:18.924435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff458c-cea7-45c7-b145-114e3fb27670 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:18.925928 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e9ff458c-cea7-45c7-b145-114e3fb27670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:18.941076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff458c-cea7-45c7-b145-114e3fb27670 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:18.945766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeaf601-f8c9-4179-8e15-ce5c04749366 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:18.946436 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeaf601-f8c9-4179-8e15-ce5c04749366 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:18.946719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeaf601-f8c9-4179-8e15-ce5c04749366 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:18.947165 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeaf601-f8c9-4179-8e15-ce5c04749366 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:18.957750 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeaf601-f8c9-4179-8e15-ce5c04749366 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:18.959208 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeaf601-f8c9-4179-8e15-ce5c04749366 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:18.969714 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1100/2193] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:59:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:59:23.992899 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6320b71-49cd-4b4e-acd3-d801990f616e service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:24.061213 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6320b71-49cd-4b4e-acd3-d801990f616e service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:24.065761 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8ea68-f8bc-4c9e-9478-e220fe10d108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:24.066394 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8ea68-f8bc-4c9e-9478-e220fe10d108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:24.066674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8ea68-f8bc-4c9e-9478-e220fe10d108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:24.066951 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8ea68-f8bc-4c9e-9478-e220fe10d108 admin admin] PATH_INFO: `/v3/users/dcc31a9543cf459b87b42914c1f6fb44` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:24.068119 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8ea68-f8bc-4c9e-9478-e220fe10d108 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcc31a9543cf459b87b42914c1f6fb44)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:24.070034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8ea68-f8bc-4c9e-9478-e220fe10d108 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:24.118057 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5ed8ea68-f8bc-4c9e-9478-e220fe10d108 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcc31a9543cf459b87b42914c1f6fb44'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:24.125960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5ed8ea68-f8bc-4c9e-9478-e220fe10d108 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcc31a9543cf459b87b42914c1f6fb44'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:24.139078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5ed8ea68-f8bc-4c9e-9478-e220fe10d108 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcc31a9543cf459b87b42914c1f6fb44'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:24.145768 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5ed8ea68-f8bc-4c9e-9478-e220fe10d108 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcc31a9543cf459b87b42914c1f6fb44'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:24.146432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5ed8ea68-f8bc-4c9e-9478-e220fe10d108 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcc31a9543cf459b87b42914c1f6fb44'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:24.152834 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1094/2194] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:59:23 2022] DELETE /identity/v3/users/dcc31a9543cf459b87b42914c1f6fb44 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:59:24.158980 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd88de-7f27-4a2a-b471-b8deebeab4b7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:24.159552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd88de-7f27-4a2a-b471-b8deebeab4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:24.159552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd88de-7f27-4a2a-b471-b8deebeab4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:24.179943 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-64fd88de-7f27-4a2a-b471-b8deebeab4b7 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:24.181166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-64fd88de-7f27-4a2a-b471-b8deebeab4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:24.239643 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1101/2195] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:24.250039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8ea68-f8bc-4c9e-9478-e220fe10d108 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:24.251445 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5ed8ea68-f8bc-4c9e-9478-e220fe10d108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:24.308451 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8ea68-f8bc-4c9e-9478-e220fe10d108 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:24.312857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96970973-3a18-4282-ba5e-1761c6265c19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:24.313465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96970973-3a18-4282-ba5e-1761c6265c19 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:24.313747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96970973-3a18-4282-ba5e-1761c6265c19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:24.314021 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96970973-3a18-4282-ba5e-1761c6265c19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:24.323077 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96970973-3a18-4282-ba5e-1761c6265c19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:24.324720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96970973-3a18-4282-ba5e-1761c6265c19 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:24.337811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1095/2196] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:59:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:59:24.623822 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd88de-7f27-4a2a-b471-b8deebeab4b7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:24.625774 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-64fd88de-7f27-4a2a-b471-b8deebeab4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:24.640662 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd88de-7f27-4a2a-b471-b8deebeab4b7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:24.646394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b269720-774d-472b-aca9-4e386af51356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:24.647124 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b269720-774d-472b-aca9-4e386af51356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:24.647482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b269720-774d-472b-aca9-4e386af51356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:24.647820 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b269720-774d-472b-aca9-4e386af51356 admin admin] PATH_INFO: `/v3/projects/d5382ca4bd054fec89a6532a732aa72e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:24.649032 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b269720-774d-472b-aca9-4e386af51356 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5382ca4bd054fec89a6532a732aa72e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:24.650917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b269720-774d-472b-aca9-4e386af51356 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:24.702431 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3b269720-774d-472b-aca9-4e386af51356 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5382ca4bd054fec89a6532a732aa72e', 'request_id': 'req-3b269720-774d-472b-aca9-4e386af51356'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:24.708297 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3b269720-774d-472b-aca9-4e386af51356 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5382ca4bd054fec89a6532a732aa72e', 'request_id': 'req-3b269720-774d-472b-aca9-4e386af51356'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:24.716008 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3b269720-774d-472b-aca9-4e386af51356 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5382ca4bd054fec89a6532a732aa72e', 'request_id': 'req-3b269720-774d-472b-aca9-4e386af51356'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:24.728397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3b269720-774d-472b-aca9-4e386af51356 admin admin] The token cache is being invalidate because project d5382ca4bd054fec89a6532a732aa72e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:59:24.728684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3b269720-774d-472b-aca9-4e386af51356 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:24.729331 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3b269720-774d-472b-aca9-4e386af51356 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:24.730816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1102/2197] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:59:24 2022] DELETE /identity/v3/projects/d5382ca4bd054fec89a6532a732aa72e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:59:24.735961 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96970973-3a18-4282-ba5e-1761c6265c19 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:24.746613 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96970973-3a18-4282-ba5e-1761c6265c19 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:24.766709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:24.806716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e64941-b55d-4489-9984-3b1b2d87d2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:24.807350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e64941-b55d-4489-9984-3b1b2d87d2a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:24.807628 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e64941-b55d-4489-9984-3b1b2d87d2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:24.807901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e64941-b55d-4489-9984-3b1b2d87d2a5 admin admin] PATH_INFO: `/v3/users/df9b20d3c97b4257a8c99398efdf5b6d` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:24.809042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e64941-b55d-4489-9984-3b1b2d87d2a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df9b20d3c97b4257a8c99398efdf5b6d)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:24.811567 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e64941-b55d-4489-9984-3b1b2d87d2a5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:24.859649 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-37e64941-b55d-4489-9984-3b1b2d87d2a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df9b20d3c97b4257a8c99398efdf5b6d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:24.867012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-37e64941-b55d-4489-9984-3b1b2d87d2a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df9b20d3c97b4257a8c99398efdf5b6d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:24.880259 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-37e64941-b55d-4489-9984-3b1b2d87d2a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df9b20d3c97b4257a8c99398efdf5b6d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:24.887564 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-37e64941-b55d-4489-9984-3b1b2d87d2a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df9b20d3c97b4257a8c99398efdf5b6d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:24.888587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-37e64941-b55d-4489-9984-3b1b2d87d2a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df9b20d3c97b4257a8c99398efdf5b6d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:24.894653 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1096/2198] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 08:59:24 2022] DELETE /identity/v3/users/df9b20d3c97b4257a8c99398efdf5b6d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:59:25.171015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b269720-774d-472b-aca9-4e386af51356 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:25.173156 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3b269720-774d-472b-aca9-4e386af51356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:25.194136 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b269720-774d-472b-aca9-4e386af51356 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:25.251849 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d402c0-92dd-431e-99f1-ce99d3d37bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:25.252511 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d402c0-92dd-431e-99f1-ce99d3d37bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:25.252791 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d402c0-92dd-431e-99f1-ce99d3d37bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:25.253068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d402c0-92dd-431e-99f1-ce99d3d37bc7 admin admin] PATH_INFO: `/v3/projects/d05d11a2732b47feaf4210ab47b9fd60` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:25.254463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d402c0-92dd-431e-99f1-ce99d3d37bc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d05d11a2732b47feaf4210ab47b9fd60)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:25.256394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d402c0-92dd-431e-99f1-ce99d3d37bc7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:25.268481 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:25.304844 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-31d402c0-92dd-431e-99f1-ce99d3d37bc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd05d11a2732b47feaf4210ab47b9fd60', 'request_id': 'req-31d402c0-92dd-431e-99f1-ce99d3d37bc7'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:25.310210 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-31d402c0-92dd-431e-99f1-ce99d3d37bc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd05d11a2732b47feaf4210ab47b9fd60', 'request_id': 'req-31d402c0-92dd-431e-99f1-ce99d3d37bc7'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:25.317893 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-31d402c0-92dd-431e-99f1-ce99d3d37bc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd05d11a2732b47feaf4210ab47b9fd60', 'request_id': 'req-31d402c0-92dd-431e-99f1-ce99d3d37bc7'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:25.329426 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-31d402c0-92dd-431e-99f1-ce99d3d37bc7 admin admin] The token cache is being invalidate because project d05d11a2732b47feaf4210ab47b9fd60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:59:25.329715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-31d402c0-92dd-431e-99f1-ce99d3d37bc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:25.330352 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-31d402c0-92dd-431e-99f1-ce99d3d37bc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:59:25.331844 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1103/2199] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 08:59:25 2022] DELETE /identity/v3/projects/d05d11a2732b47feaf4210ab47b9fd60 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:59:25.337919 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79125b8-3f04-46b0-9faa-f9450ae6ac18 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:25.338412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79125b8-3f04-46b0-9faa-f9450ae6ac18 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:25.338603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79125b8-3f04-46b0-9faa-f9450ae6ac18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:25.359443 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d79125b8-3f04-46b0-9faa-f9450ae6ac18 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:25.360677 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d79125b8-3f04-46b0-9faa-f9450ae6ac18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:25.423488 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1097/2200] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:25.431676 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def806a1-9064-4dda-aac8-3583767484f3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:25.431957 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def806a1-9064-4dda-aac8-3583767484f3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:25.432274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def806a1-9064-4dda-aac8-3583767484f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:25.453080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-def806a1-9064-4dda-aac8-3583767484f3 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:25.454381 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-def806a1-9064-4dda-aac8-3583767484f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:25.462293 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1104/2201] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:59:25.467441 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79125b8-3f04-46b0-9faa-f9450ae6ac18 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:25.477249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79125b8-3f04-46b0-9faa-f9450ae6ac18 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:25.482980 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298de0b8-f1a4-42ab-8aaf-80d01867683d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:25.483847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298de0b8-f1a4-42ab-8aaf-80d01867683d admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:25.484154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298de0b8-f1a4-42ab-8aaf-80d01867683d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:25.484428 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298de0b8-f1a4-42ab-8aaf-80d01867683d admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:25.485084 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298de0b8-f1a4-42ab-8aaf-80d01867683d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:59:25.485390 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298de0b8-f1a4-42ab-8aaf-80d01867683d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:25.487053 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298de0b8-f1a4-42ab-8aaf-80d01867683d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:25.502561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1098/2202] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 08:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:59:25.504249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def806a1-9064-4dda-aac8-3583767484f3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:25.513703 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def806a1-9064-4dda-aac8-3583767484f3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:25.518705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9480bd-129a-4bb0-b55f-f0b45d1ac586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:25.519342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9480bd-129a-4bb0-b55f-f0b45d1ac586 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:25.519626 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9480bd-129a-4bb0-b55f-f0b45d1ac586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:25.519921 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9480bd-129a-4bb0-b55f-f0b45d1ac586 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:25.520640 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9480bd-129a-4bb0-b55f-f0b45d1ac586 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:25.522593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9480bd-129a-4bb0-b55f-f0b45d1ac586 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:25.552390 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1105/2203] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 08:59:25 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:59:25.556640 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298de0b8-f1a4-42ab-8aaf-80d01867683d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:25.566557 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298de0b8-f1a4-42ab-8aaf-80d01867683d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:25.595655 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:25.627558 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d143c2-6707-455e-9660-2e586a046a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:25.628195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d143c2-6707-455e-9660-2e586a046a64 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:25.628734 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d143c2-6707-455e-9660-2e586a046a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:25.629006 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d143c2-6707-455e-9660-2e586a046a64 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:25.629634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d143c2-6707-455e-9660-2e586a046a64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:25.631533 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d143c2-6707-455e-9660-2e586a046a64 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:25.656903 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1099/2204] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 08:59:25 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:59:25.660987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9480bd-129a-4bb0-b55f-f0b45d1ac586 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:25.672583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9480bd-129a-4bb0-b55f-f0b45d1ac586 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:25.678291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67f303-1d27-44cd-8f34-b7ca948e5f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:25.678896 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67f303-1d27-44cd-8f34-b7ca948e5f9d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:25.679175 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67f303-1d27-44cd-8f34-b7ca948e5f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:25.679445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67f303-1d27-44cd-8f34-b7ca948e5f9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:25.680109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67f303-1d27-44cd-8f34-b7ca948e5f9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:25.681694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67f303-1d27-44cd-8f34-b7ca948e5f9d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:25.692155 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1106/2205] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:59:25.696592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d143c2-6707-455e-9660-2e586a046a64 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:25.708573 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d143c2-6707-455e-9660-2e586a046a64 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:25.713276 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0aae81-379c-4322-9b9f-39401f1f28fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:25.713878 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aae81-379c-4322-9b9f-39401f1f28fb admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:25.714153 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aae81-379c-4322-9b9f-39401f1f28fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:25.714425 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aae81-379c-4322-9b9f-39401f1f28fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:25.715063 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0aae81-379c-4322-9b9f-39401f1f28fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:25.717544 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0aae81-379c-4322-9b9f-39401f1f28fb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:25.728556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1100/2206] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 08:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:59:25.732869 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67f303-1d27-44cd-8f34-b7ca948e5f9d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:25.744233 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67f303-1d27-44cd-8f34-b7ca948e5f9d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:25.749147 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623a849-95f8-40ac-adcc-7fe5a1d85910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:25.749803 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623a849-95f8-40ac-adcc-7fe5a1d85910 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:25.750084 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623a849-95f8-40ac-adcc-7fe5a1d85910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:25.750359 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623a849-95f8-40ac-adcc-7fe5a1d85910 admin admin] PATH_INFO: `/v3/projects/c1582baa8794449c8723823900a9773f/users/d679247dd043433aa29c8fda0885ea6a/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:25.767516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623a849-95f8-40ac-adcc-7fe5a1d85910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1582baa8794449c8723823900a9773f, user_id=d679247dd043433aa29c8fda0885ea6a, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:25.770718 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623a849-95f8-40ac-adcc-7fe5a1d85910 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:25.784754 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1107/2207] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 08:59:25 2022] PUT /identity/v3/projects/c1582baa8794449c8723823900a9773f/users/d679247dd043433aa29c8fda0885ea6a/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:59:25.797586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f242a9b-19bf-4bcd-8961-33c4804c864e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:25.797865 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f242a9b-19bf-4bcd-8961-33c4804c864e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:25.798162 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f242a9b-19bf-4bcd-8961-33c4804c864e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:25.827136 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-0f242a9b-19bf-4bcd-8961-33c4804c864e None None] MFA Rules not processed for user `d679247dd043433aa29c8fda0885ea6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:25.828303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0f242a9b-19bf-4bcd-8961-33c4804c864e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:25.896754 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1101/2208] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:59:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:59:25.908908 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623a849-95f8-40ac-adcc-7fe5a1d85910 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:59:25.910603 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0623a849-95f8-40ac-adcc-7fe5a1d85910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:25.973687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623a849-95f8-40ac-adcc-7fe5a1d85910 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:59:25.977252 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edeea0a-4351-4394-86e1-cc180e4cc600 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5979387f7aa54619b8d429368c306cc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:59:25.977857 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edeea0a-4351-4394-86e1-cc180e4cc600 service glance] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:25.978127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edeea0a-4351-4394-86e1-cc180e4cc600 service glance] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:25.978435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edeea0a-4351-4394-86e1-cc180e4cc600 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:25.987380 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edeea0a-4351-4394-86e1-cc180e4cc600 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:59:25.988881 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edeea0a-4351-4394-86e1-cc180e4cc600 service glance] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:59:25.999065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1108/2209] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 08:59:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:59:35.893822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:35.994276 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:59:43.568134 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0a33e-533d-468a-8231-296f2ebee164 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:59:43.568791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0a33e-533d-468a-8231-296f2ebee164 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:59:43.569087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0a33e-533d-468a-8231-296f2ebee164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:59:43.589815 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-dac0a33e-533d-468a-8231-296f2ebee164 None None] MFA Rules not processed for user `46b3aa99a5e84ef2a8df193628feb0d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:59:43.590970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dac0a33e-533d-468a-8231-296f2ebee164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:59:43.657746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1102/2210] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 08:59:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:59:53.649298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=25 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:06.398524 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edeea0a-4351-4394-86e1-cc180e4cc600 service glance] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:06.399935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8edeea0a-4351-4394-86e1-cc180e4cc600 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:06.463943 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edeea0a-4351-4394-86e1-cc180e4cc600 service glance] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:06.467702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85cbd8-dca7-46e7-8845-a349be1b0da6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:06.468350 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85cbd8-dca7-46e7-8845-a349be1b0da6 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:06.468698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85cbd8-dca7-46e7-8845-a349be1b0da6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:06.469503 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85cbd8-dca7-46e7-8845-a349be1b0da6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:06.471281 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bd85cbd8-dca7-46e7-8845-a349be1b0da6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:06.537694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85cbd8-dca7-46e7-8845-a349be1b0da6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:06.539133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85cbd8-dca7-46e7-8845-a349be1b0da6 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:06.549263 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1109/2211] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:06.993644 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0a33e-533d-468a-8231-296f2ebee164 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:07.060012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0a33e-533d-468a-8231-296f2ebee164 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:07.064610 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c10b5-5efc-41b0-a509-5803bd8c98a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:07.065254 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c10b5-5efc-41b0-a509-5803bd8c98a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:07.065534 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c10b5-5efc-41b0-a509-5803bd8c98a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:07.065808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c10b5-5efc-41b0-a509-5803bd8c98a0 admin admin] PATH_INFO: `/v3/users/d679247dd043433aa29c8fda0885ea6a` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:07.067026 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c10b5-5efc-41b0-a509-5803bd8c98a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d679247dd043433aa29c8fda0885ea6a)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:07.068998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c10b5-5efc-41b0-a509-5803bd8c98a0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:07.119857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d29c10b5-5efc-41b0-a509-5803bd8c98a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd679247dd043433aa29c8fda0885ea6a'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:07.128266 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d29c10b5-5efc-41b0-a509-5803bd8c98a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd679247dd043433aa29c8fda0885ea6a'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:07.143120 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d29c10b5-5efc-41b0-a509-5803bd8c98a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd679247dd043433aa29c8fda0885ea6a'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:07.150890 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d29c10b5-5efc-41b0-a509-5803bd8c98a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd679247dd043433aa29c8fda0885ea6a'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:07.151637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d29c10b5-5efc-41b0-a509-5803bd8c98a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd679247dd043433aa29c8fda0885ea6a'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:07.158517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1103/2212] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:00:06 2022] DELETE /identity/v3/users/d679247dd043433aa29c8fda0885ea6a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:00:07.163021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f326c6d-1e52-4ed5-8459-1d7ca13c3f9f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:07.163312 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f326c6d-1e52-4ed5-8459-1d7ca13c3f9f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:07.163589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f326c6d-1e52-4ed5-8459-1d7ca13c3f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:07.191379 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6f326c6d-1e52-4ed5-8459-1d7ca13c3f9f None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:07.193168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6f326c6d-1e52-4ed5-8459-1d7ca13c3f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:07.265435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1110/2213] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:07.275763 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c10b5-5efc-41b0-a509-5803bd8c98a0 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:07.277215 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d29c10b5-5efc-41b0-a509-5803bd8c98a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:07.340282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c10b5-5efc-41b0-a509-5803bd8c98a0 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:07.344681 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995324d-32f8-423c-a6a4-b3e1181e75de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:07.345282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995324d-32f8-423c-a6a4-b3e1181e75de service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:07.345559 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995324d-32f8-423c-a6a4-b3e1181e75de service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:07.345827 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995324d-32f8-423c-a6a4-b3e1181e75de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:07.355113 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995324d-32f8-423c-a6a4-b3e1181e75de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:07.356699 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995324d-32f8-423c-a6a4-b3e1181e75de service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:07.367360 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1104/2214] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:07.665789 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f326c6d-1e52-4ed5-8459-1d7ca13c3f9f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:07.667485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6f326c6d-1e52-4ed5-8459-1d7ca13c3f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:07.680294 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f326c6d-1e52-4ed5-8459-1d7ca13c3f9f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:07.685411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845674d9-9049-4bdb-beb8-a628ec2e2094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:07.686061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845674d9-9049-4bdb-beb8-a628ec2e2094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:07.686343 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845674d9-9049-4bdb-beb8-a628ec2e2094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:07.686747 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845674d9-9049-4bdb-beb8-a628ec2e2094 admin admin] PATH_INFO: `/v3/projects/c1582baa8794449c8723823900a9773f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:07.688759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845674d9-9049-4bdb-beb8-a628ec2e2094 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1582baa8794449c8723823900a9773f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:07.690726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845674d9-9049-4bdb-beb8-a628ec2e2094 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:07.744160 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:07.748310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-845674d9-9049-4bdb-beb8-a628ec2e2094 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1582baa8794449c8723823900a9773f', 'request_id': 'req-845674d9-9049-4bdb-beb8-a628ec2e2094'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:07.754819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-845674d9-9049-4bdb-beb8-a628ec2e2094 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1582baa8794449c8723823900a9773f', 'request_id': 'req-845674d9-9049-4bdb-beb8-a628ec2e2094'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:07.762836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-845674d9-9049-4bdb-beb8-a628ec2e2094 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1582baa8794449c8723823900a9773f', 'request_id': 'req-845674d9-9049-4bdb-beb8-a628ec2e2094'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:07.776618 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-845674d9-9049-4bdb-beb8-a628ec2e2094 admin admin] The token cache is being invalidate because project c1582baa8794449c8723823900a9773f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:00:07.776911 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-845674d9-9049-4bdb-beb8-a628ec2e2094 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:07.777573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-845674d9-9049-4bdb-beb8-a628ec2e2094 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:07.779345 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1111/2215] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:00:07 2022] DELETE /identity/v3/projects/c1582baa8794449c8723823900a9773f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:00:07.784851 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99162cde-8cf7-4e6f-92a7-3caebec3b400 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:07.785140 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99162cde-8cf7-4e6f-92a7-3caebec3b400 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:07.785417 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99162cde-8cf7-4e6f-92a7-3caebec3b400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:07.796246 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845674d9-9049-4bdb-beb8-a628ec2e2094 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:07.797770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-845674d9-9049-4bdb-beb8-a628ec2e2094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:07.807359 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-99162cde-8cf7-4e6f-92a7-3caebec3b400 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:07.808599 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-99162cde-8cf7-4e6f-92a7-3caebec3b400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:07.827710 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:07.871023 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845674d9-9049-4bdb-beb8-a628ec2e2094 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:07.877421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdca86f-934d-441e-a417-bdc5389b12da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:07.878134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdca86f-934d-441e-a417-bdc5389b12da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:07.878425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdca86f-934d-441e-a417-bdc5389b12da admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:07.878859 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdca86f-934d-441e-a417-bdc5389b12da admin admin] PATH_INFO: `/v3/users/f701cf08283b4ee78b9eaa502890507f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:07.880018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdca86f-934d-441e-a417-bdc5389b12da admin admin] RBAC: Authorizing `identity:delete_user(user_id=f701cf08283b4ee78b9eaa502890507f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:07.882500 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdca86f-934d-441e-a417-bdc5389b12da admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:07.890521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1105/2216] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:07.898430 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07618cf5-dec3-4d2a-9da4-f18df49c9d4b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:07.898717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07618cf5-dec3-4d2a-9da4-f18df49c9d4b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:07.898995 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07618cf5-dec3-4d2a-9da4-f18df49c9d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:07.921576 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-07618cf5-dec3-4d2a-9da4-f18df49c9d4b None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:07.923781 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-07618cf5-dec3-4d2a-9da4-f18df49c9d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:07.932221 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1106/2217] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:07.936303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07618cf5-dec3-4d2a-9da4-f18df49c9d4b None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:07.942736 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5fdca86f-934d-441e-a417-bdc5389b12da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f701cf08283b4ee78b9eaa502890507f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:07.943390 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5fdca86f-934d-441e-a417-bdc5389b12da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f701cf08283b4ee78b9eaa502890507f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:07.947807 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07618cf5-dec3-4d2a-9da4-f18df49c9d4b None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:07.952533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5fdca86f-934d-441e-a417-bdc5389b12da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f701cf08283b4ee78b9eaa502890507f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:07.955237 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0114eae-bdea-45e6-a35d-d0b0791d4ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:07.955934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0114eae-bdea-45e6-a35d-d0b0791d4ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:07.956860 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0114eae-bdea-45e6-a35d-d0b0791d4ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:07.957165 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0114eae-bdea-45e6-a35d-d0b0791d4ef7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:07.957695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0114eae-bdea-45e6-a35d-d0b0791d4ef7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:00:07.957999 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0114eae-bdea-45e6-a35d-d0b0791d4ef7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:07.960795 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0114eae-bdea-45e6-a35d-d0b0791d4ef7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:07.961118 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5fdca86f-934d-441e-a417-bdc5389b12da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f701cf08283b4ee78b9eaa502890507f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:07.974177 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1107/2218] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:00:07.977874 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5fdca86f-934d-441e-a417-bdc5389b12da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f701cf08283b4ee78b9eaa502890507f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:07.978578 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0114eae-bdea-45e6-a35d-d0b0791d4ef7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:07.980328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d0114eae-bdea-45e6-a35d-d0b0791d4ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:07.988430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1112/2219] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:00:07 2022] DELETE /identity/v3/users/f701cf08283b4ee78b9eaa502890507f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:00:07.991586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e517e-1768-40aa-a73a-33d9f73afc5e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:07.991987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e517e-1768-40aa-a73a-33d9f73afc5e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:07.992359 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e517e-1768-40aa-a73a-33d9f73afc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:07.995093 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0114eae-bdea-45e6-a35d-d0b0791d4ef7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:08.013920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-157e517e-1768-40aa-a73a-33d9f73afc5e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:08.015144 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-157e517e-1768-40aa-a73a-33d9f73afc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:08.073095 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed546448-6b97-4c1a-adf1-86932cb36fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:08.074091 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed546448-6b97-4c1a-adf1-86932cb36fda admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:08.074376 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed546448-6b97-4c1a-adf1-86932cb36fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:08.074740 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed546448-6b97-4c1a-adf1-86932cb36fda admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:08.075304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed546448-6b97-4c1a-adf1-86932cb36fda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:08.076552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:08.083465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed546448-6b97-4c1a-adf1-86932cb36fda admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:08.099212 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1113/2220] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:00:08.107616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e517e-1768-40aa-a73a-33d9f73afc5e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:08.109483 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-157e517e-1768-40aa-a73a-33d9f73afc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:08.120169 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1108/2221] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:00:07 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:00:08.124408 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed546448-6b97-4c1a-adf1-86932cb36fda admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:08.182940 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e517e-1768-40aa-a73a-33d9f73afc5e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:08.205644 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed546448-6b97-4c1a-adf1-86932cb36fda admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:08.210616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a91add-8b6f-417f-8804-bf6b53fe67fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:08.211224 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a91add-8b6f-417f-8804-bf6b53fe67fe admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:08.211557 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a91add-8b6f-417f-8804-bf6b53fe67fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:08.211838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a91add-8b6f-417f-8804-bf6b53fe67fe admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:08.212763 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a91add-8b6f-417f-8804-bf6b53fe67fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:08.214554 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a91add-8b6f-417f-8804-bf6b53fe67fe admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:08.241514 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1109/2222] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:00:08 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:00:08.241770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451be79-59c3-4f34-b685-79fc98e7789b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:08.242353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451be79-59c3-4f34-b685-79fc98e7789b service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:08.242631 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451be79-59c3-4f34-b685-79fc98e7789b service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:08.242927 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451be79-59c3-4f34-b685-79fc98e7789b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:08.243033 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a91add-8b6f-417f-8804-bf6b53fe67fe admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:08.253880 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451be79-59c3-4f34-b685-79fc98e7789b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:08.255408 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a91add-8b6f-417f-8804-bf6b53fe67fe admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:08.255542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451be79-59c3-4f34-b685-79fc98e7789b service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:08.260925 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d43787-32e3-4b14-899d-3604012dd6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:08.262418 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d43787-32e3-4b14-899d-3604012dd6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:08.262418 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d43787-32e3-4b14-899d-3604012dd6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:08.262418 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d43787-32e3-4b14-899d-3604012dd6d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:08.262797 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d43787-32e3-4b14-899d-3604012dd6d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:08.264332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d43787-32e3-4b14-899d-3604012dd6d9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:08.268673 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1114/2223] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:08.281545 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1110/2224] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:00:08.285651 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451be79-59c3-4f34-b685-79fc98e7789b service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:08.296616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451be79-59c3-4f34-b685-79fc98e7789b service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:08.302891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399abee7-99d7-416f-b96e-acd128bd74dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:08.303305 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399abee7-99d7-416f-b96e-acd128bd74dc admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:08.303587 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399abee7-99d7-416f-b96e-acd128bd74dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:08.303855 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399abee7-99d7-416f-b96e-acd128bd74dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:08.304536 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399abee7-99d7-416f-b96e-acd128bd74dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:08.306380 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399abee7-99d7-416f-b96e-acd128bd74dc admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:08.326542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1115/2225] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:00:08.328934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d43787-32e3-4b14-899d-3604012dd6d9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:08.344303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d43787-32e3-4b14-899d-3604012dd6d9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:08.350318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0f91d-653e-4aeb-afdc-3d0244d80f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:08.351339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0f91d-653e-4aeb-afdc-3d0244d80f18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:08.351823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0f91d-653e-4aeb-afdc-3d0244d80f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:08.352350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0f91d-653e-4aeb-afdc-3d0244d80f18 admin admin] PATH_INFO: `/v3/projects/804a47ef4a8a410ca25640b1986a5c39/users/614c7c155a814d2b93661c7feb920883/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:08.355312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:08.373985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0f91d-653e-4aeb-afdc-3d0244d80f18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=804a47ef4a8a410ca25640b1986a5c39, user_id=614c7c155a814d2b93661c7feb920883, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:08.377320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0f91d-653e-4aeb-afdc-3d0244d80f18 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:08.399171 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1111/2226] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:00:08 2022] PUT /identity/v3/projects/804a47ef4a8a410ca25640b1986a5c39/users/614c7c155a814d2b93661c7feb920883/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:00:08.410109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed361639-5a25-4f61-94b8-f36e2495f03e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:08.410393 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed361639-5a25-4f61-94b8-f36e2495f03e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:08.410670 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed361639-5a25-4f61-94b8-f36e2495f03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:08.451748 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ed361639-5a25-4f61-94b8-f36e2495f03e None None] MFA Rules not processed for user `614c7c155a814d2b93661c7feb920883`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:08.453433 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ed361639-5a25-4f61-94b8-f36e2495f03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:08.527863 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1116/2227] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:00:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:08.539789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0f91d-653e-4aeb-afdc-3d0244d80f18 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:08.541761 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4bb0f91d-653e-4aeb-afdc-3d0244d80f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:08.565657 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed361639-5a25-4f61-94b8-f36e2495f03e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:08.567197 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ed361639-5a25-4f61-94b8-f36e2495f03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:08.618982 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:08.659508 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed361639-5a25-4f61-94b8-f36e2495f03e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:08.665463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21c912-64bc-4ecc-af17-904a2ddc371c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:08.666110 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21c912-64bc-4ecc-af17-904a2ddc371c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:08.666664 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21c912-64bc-4ecc-af17-904a2ddc371c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:08.666664 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21c912-64bc-4ecc-af17-904a2ddc371c admin admin] PATH_INFO: `/v3/projects/ae769ec5950844b5940fbc977f50ca5b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:08.668740 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0f91d-653e-4aeb-afdc-3d0244d80f18 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:08.669570 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21c912-64bc-4ecc-af17-904a2ddc371c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae769ec5950844b5940fbc977f50ca5b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:08.671480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21c912-64bc-4ecc-af17-904a2ddc371c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:08.679784 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b183e1-f2bc-46f9-bcf9-a945e2e13f03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:08.680401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b183e1-f2bc-46f9-bcf9-a945e2e13f03 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:08.680677 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b183e1-f2bc-46f9-bcf9-a945e2e13f03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:08.680942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b183e1-f2bc-46f9-bcf9-a945e2e13f03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:08.710984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b183e1-f2bc-46f9-bcf9-a945e2e13f03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:08.712748 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b183e1-f2bc-46f9-bcf9-a945e2e13f03 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:08.731275 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1112/2228] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:08.772129 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5d21c912-64bc-4ecc-af17-904a2ddc371c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae769ec5950844b5940fbc977f50ca5b', 'request_id': 'req-5d21c912-64bc-4ecc-af17-904a2ddc371c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:08.781410 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5d21c912-64bc-4ecc-af17-904a2ddc371c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae769ec5950844b5940fbc977f50ca5b', 'request_id': 'req-5d21c912-64bc-4ecc-af17-904a2ddc371c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:08.791994 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5d21c912-64bc-4ecc-af17-904a2ddc371c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae769ec5950844b5940fbc977f50ca5b', 'request_id': 'req-5d21c912-64bc-4ecc-af17-904a2ddc371c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:08.807059 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5d21c912-64bc-4ecc-af17-904a2ddc371c admin admin] The token cache is being invalidate because project ae769ec5950844b5940fbc977f50ca5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:00:08.807572 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5d21c912-64bc-4ecc-af17-904a2ddc371c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:08.808518 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5d21c912-64bc-4ecc-af17-904a2ddc371c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:08.815242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1117/2229] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:00:08 2022] DELETE /identity/v3/projects/ae769ec5950844b5940fbc977f50ca5b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:00:08.816662 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83057b3-5ca5-49e0-9064-33e5f93407ca None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:08.816949 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83057b3-5ca5-49e0-9064-33e5f93407ca None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:08.817223 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83057b3-5ca5-49e0-9064-33e5f93407ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:08.859529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d83057b3-5ca5-49e0-9064-33e5f93407ca None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:08.862086 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d83057b3-5ca5-49e0-9064-33e5f93407ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:08.964507 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1113/2230] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:09.035694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c42ab-d2fd-493d-9c65-f14f5d5ba2bf None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:09.036357 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c42ab-d2fd-493d-9c65-f14f5d5ba2bf None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:09.036871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c42ab-d2fd-493d-9c65-f14f5d5ba2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:09.066190 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-272c42ab-d2fd-493d-9c65-f14f5d5ba2bf None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:09.067499 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-272c42ab-d2fd-493d-9c65-f14f5d5ba2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:09.077780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1118/2231] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:09.081999 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83057b3-5ca5-49e0-9064-33e5f93407ca None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:09.097509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83057b3-5ca5-49e0-9064-33e5f93407ca None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:09.104402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c107ec-8d84-4a77-92cf-bc3c541a02ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:09.105399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c107ec-8d84-4a77-92cf-bc3c541a02ad admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:09.105741 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c107ec-8d84-4a77-92cf-bc3c541a02ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:09.106074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c107ec-8d84-4a77-92cf-bc3c541a02ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:09.107153 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c107ec-8d84-4a77-92cf-bc3c541a02ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:00:09.107524 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c107ec-8d84-4a77-92cf-bc3c541a02ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:09.110064 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c107ec-8d84-4a77-92cf-bc3c541a02ad admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:09.125872 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1114/2232] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:00:09.130280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c42ab-d2fd-493d-9c65-f14f5d5ba2bf None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:09.146661 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c42ab-d2fd-493d-9c65-f14f5d5ba2bf None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:09.152253 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb63bb-a80d-422f-807f-32a3c9a12225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:09.153567 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb63bb-a80d-422f-807f-32a3c9a12225 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:09.153946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb63bb-a80d-422f-807f-32a3c9a12225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:09.154225 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb63bb-a80d-422f-807f-32a3c9a12225 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:09.155092 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb63bb-a80d-422f-807f-32a3c9a12225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:09.157436 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb63bb-a80d-422f-807f-32a3c9a12225 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:09.192123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1119/2233] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:00:09 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:00:09.196555 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c107ec-8d84-4a77-92cf-bc3c541a02ad admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:09.216348 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c107ec-8d84-4a77-92cf-bc3c541a02ad admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:09.291290 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49188fb9-b051-4f38-8342-e2df1ea1e1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:09.291961 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49188fb9-b051-4f38-8342-e2df1ea1e1c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:09.292279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49188fb9-b051-4f38-8342-e2df1ea1e1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:09.292556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49188fb9-b051-4f38-8342-e2df1ea1e1c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:09.293220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49188fb9-b051-4f38-8342-e2df1ea1e1c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:09.295373 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49188fb9-b051-4f38-8342-e2df1ea1e1c8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:09.325481 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1115/2234] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:00:09 2022] POST /identity/v3/users => generated 287 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:00:09.330222 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb63bb-a80d-422f-807f-32a3c9a12225 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:09.342602 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb63bb-a80d-422f-807f-32a3c9a12225 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:09.347609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed72d2c-dc9e-44eb-8858-c751904e4cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:09.348330 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed72d2c-dc9e-44eb-8858-c751904e4cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:09.348676 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed72d2c-dc9e-44eb-8858-c751904e4cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:09.349014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed72d2c-dc9e-44eb-8858-c751904e4cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:09.349857 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed72d2c-dc9e-44eb-8858-c751904e4cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:09.351507 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed72d2c-dc9e-44eb-8858-c751904e4cf0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:09.366130 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1120/2235] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:00:09.370265 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49188fb9-b051-4f38-8342-e2df1ea1e1c8 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:09.377800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:09.387189 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49188fb9-b051-4f38-8342-e2df1ea1e1c8 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:09.393865 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a793e-325e-4be5-a339-043b1df43863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:09.394571 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a793e-325e-4be5-a339-043b1df43863 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:09.394923 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a793e-325e-4be5-a339-043b1df43863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:09.395264 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a793e-325e-4be5-a339-043b1df43863 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:09.395992 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a793e-325e-4be5-a339-043b1df43863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:09.398532 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a793e-325e-4be5-a339-043b1df43863 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:09.412791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1116/2236] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:00:09.416929 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed72d2c-dc9e-44eb-8858-c751904e4cf0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:09.431573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed72d2c-dc9e-44eb-8858-c751904e4cf0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:09.437090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f80b3-b1a9-4d8b-aa3d-a5f35ff77e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:09.437822 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f80b3-b1a9-4d8b-aa3d-a5f35ff77e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:09.438100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f80b3-b1a9-4d8b-aa3d-a5f35ff77e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:09.438370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f80b3-b1a9-4d8b-aa3d-a5f35ff77e64 admin admin] PATH_INFO: `/v3/projects/3d68ee6e33664bbfba3b239f16fbbaa0/users/edc56fc598e741ceab98edd3b9afa28c/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:09.454432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f80b3-b1a9-4d8b-aa3d-a5f35ff77e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d68ee6e33664bbfba3b239f16fbbaa0, user_id=edc56fc598e741ceab98edd3b9afa28c, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:09.457900 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f80b3-b1a9-4d8b-aa3d-a5f35ff77e64 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:09.473097 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1121/2237] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:00:09 2022] PUT /identity/v3/projects/3d68ee6e33664bbfba3b239f16fbbaa0/users/edc56fc598e741ceab98edd3b9afa28c/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:00:09.478110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18c91c-8253-4e09-8c8f-4245cc26e054 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:09.478398 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18c91c-8253-4e09-8c8f-4245cc26e054 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:09.478672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18c91c-8253-4e09-8c8f-4245cc26e054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:09.498479 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9e18c91c-8253-4e09-8c8f-4245cc26e054 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:09.499753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9e18c91c-8253-4e09-8c8f-4245cc26e054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:09.560620 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1117/2238] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:09.570694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f80b3-b1a9-4d8b-aa3d-a5f35ff77e64 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:09.572481 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ac1f80b3-b1a9-4d8b-aa3d-a5f35ff77e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:09.633378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f80b3-b1a9-4d8b-aa3d-a5f35ff77e64 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:09.637003 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24bbb8-79ce-4141-9a0e-7b31c71568e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:09.637606 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24bbb8-79ce-4141-9a0e-7b31c71568e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:09.637887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24bbb8-79ce-4141-9a0e-7b31c71568e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:09.638154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24bbb8-79ce-4141-9a0e-7b31c71568e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:09.647690 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24bbb8-79ce-4141-9a0e-7b31c71568e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:09.649168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24bbb8-79ce-4141-9a0e-7b31c71568e0 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:09.655954 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:09.664164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1122/2239] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:14.149439 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc61ba-20c9-4bab-a197-029de608f031 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:14.149957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc61ba-20c9-4bab-a197-029de608f031 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:14.149957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc61ba-20c9-4bab-a197-029de608f031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:14.186881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b7bc61ba-20c9-4bab-a197-029de608f031 None None] MFA Rules not processed for user `edc56fc598e741ceab98edd3b9afa28c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:14.189158 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b7bc61ba-20c9-4bab-a197-029de608f031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:14.270174 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1118/2240] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:00:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:14.274665 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24bbb8-79ce-4141-9a0e-7b31c71568e0 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:14.285574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24bbb8-79ce-4141-9a0e-7b31c71568e0 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:14.291556 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee230e1-30f2-4d3f-bea7-2ddb003a53e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:14.292176 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee230e1-30f2-4d3f-bea7-2ddb003a53e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:14.292421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee230e1-30f2-4d3f-bea7-2ddb003a53e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:14.292698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee230e1-30f2-4d3f-bea7-2ddb003a53e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:14.293401 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee230e1-30f2-4d3f-bea7-2ddb003a53e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:14.295487 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee230e1-30f2-4d3f-bea7-2ddb003a53e2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:14.327910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1123/2241] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:00:14 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:00:14.332604 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc61ba-20c9-4bab-a197-029de608f031 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:14.343736 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc61ba-20c9-4bab-a197-029de608f031 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:14.412003 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13af9a-7ce3-4c66-afe2-f2bb23c1d9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:14.412683 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13af9a-7ce3-4c66-afe2-f2bb23c1d9a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:14.412967 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13af9a-7ce3-4c66-afe2-f2bb23c1d9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:14.413274 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13af9a-7ce3-4c66-afe2-f2bb23c1d9a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:14.413898 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13af9a-7ce3-4c66-afe2-f2bb23c1d9a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:14.417681 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13af9a-7ce3-4c66-afe2-f2bb23c1d9a7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:14.442407 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1119/2242] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:00:14 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:00:14.446510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee230e1-30f2-4d3f-bea7-2ddb003a53e2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:14.457331 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee230e1-30f2-4d3f-bea7-2ddb003a53e2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:14.462603 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8e261-5810-4910-89cb-96fcab18612f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:14.463367 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8e261-5810-4910-89cb-96fcab18612f admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:14.463720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8e261-5810-4910-89cb-96fcab18612f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:14.464143 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8e261-5810-4910-89cb-96fcab18612f admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:14.464946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8e261-5810-4910-89cb-96fcab18612f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:14.466772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8e261-5810-4910-89cb-96fcab18612f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:14.478261 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1124/2243] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:00:14.482567 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13af9a-7ce3-4c66-afe2-f2bb23c1d9a7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:14.493723 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13af9a-7ce3-4c66-afe2-f2bb23c1d9a7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:14.499953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76069388-cfea-444c-b39a-2535daf2138b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:14.501277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76069388-cfea-444c-b39a-2535daf2138b admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:14.501626 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76069388-cfea-444c-b39a-2535daf2138b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:14.501880 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76069388-cfea-444c-b39a-2535daf2138b admin admin] PATH_INFO: `/v3/projects/3ddab7b8618a42f1845f4ed86ca93b6b/users/bf53ca52f58d4170a256aae15fdaf8e7/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:14.511516 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:14.519858 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76069388-cfea-444c-b39a-2535daf2138b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ddab7b8618a42f1845f4ed86ca93b6b, user_id=bf53ca52f58d4170a256aae15fdaf8e7, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:14.522873 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76069388-cfea-444c-b39a-2535daf2138b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:14.537361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1120/2244] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:00:14 2022] PUT /identity/v3/projects/3ddab7b8618a42f1845f4ed86ca93b6b/users/bf53ca52f58d4170a256aae15fdaf8e7/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:00:14.541240 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8e261-5810-4910-89cb-96fcab18612f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:14.571292 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8e261-5810-4910-89cb-96fcab18612f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:14.650197 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370a61d-4c21-4848-ab15-fa91d3c07159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:14.650921 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370a61d-4c21-4848-ab15-fa91d3c07159 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:14.651269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370a61d-4c21-4848-ab15-fa91d3c07159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:14.651608 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370a61d-4c21-4848-ab15-fa91d3c07159 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:14.652355 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370a61d-4c21-4848-ab15-fa91d3c07159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:14.654297 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370a61d-4c21-4848-ab15-fa91d3c07159 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:14.671016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1125/2245] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:00:14.673304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76069388-cfea-444c-b39a-2535daf2138b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:14.687896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76069388-cfea-444c-b39a-2535daf2138b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:14.698033 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e66e5-034e-4344-bf4a-134ee497f286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:14.699265 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e66e5-034e-4344-bf4a-134ee497f286 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:14.699432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e66e5-034e-4344-bf4a-134ee497f286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:14.699896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e66e5-034e-4344-bf4a-134ee497f286 admin admin] PATH_INFO: `/v3/projects/3ddab7b8618a42f1845f4ed86ca93b6b/users/bf53ca52f58d4170a256aae15fdaf8e7/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:14.705127 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e66e5-034e-4344-bf4a-134ee497f286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ddab7b8618a42f1845f4ed86ca93b6b, user_id=bf53ca52f58d4170a256aae15fdaf8e7, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:14.706601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e66e5-034e-4344-bf4a-134ee497f286 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:14.707463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912390de-6686-4f60-a3da-94c535afaea7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:14.707749 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912390de-6686-4f60-a3da-94c535afaea7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:14.710640 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912390de-6686-4f60-a3da-94c535afaea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:14.724697 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1121/2246] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:00:14 2022] PUT /identity/v3/projects/3ddab7b8618a42f1845f4ed86ca93b6b/users/bf53ca52f58d4170a256aae15fdaf8e7/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:00:14.736653 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-912390de-6686-4f60-a3da-94c535afaea7 None None] MFA Rules not processed for user `b11abed8d4424a1e9957c3dc9aefadd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:14.738002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-912390de-6686-4f60-a3da-94c535afaea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:14.810849 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1126/2247] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 24 09:00:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:00:14.888896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e66e5-034e-4344-bf4a-134ee497f286 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:14.891944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-469e66e5-034e-4344-bf4a-134ee497f286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:15.017908 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e66e5-034e-4344-bf4a-134ee497f286 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:15.036146 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458b1ab-7833-44be-8a97-c9b0d729d182 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46b1417d4ac24c8982b2052f4e99042a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:15.038294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458b1ab-7833-44be-8a97-c9b0d729d182 service swift] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:15.038557 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458b1ab-7833-44be-8a97-c9b0d729d182 service swift] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:15.038834 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458b1ab-7833-44be-8a97-c9b0d729d182 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:15.061058 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458b1ab-7833-44be-8a97-c9b0d729d182 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:15.062721 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458b1ab-7833-44be-8a97-c9b0d729d182 service swift] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:15.088816 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1122/2248] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 24 09:00:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:00:19.317727 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912390de-6686-4f60-a3da-94c535afaea7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:19.319189 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-912390de-6686-4f60-a3da-94c535afaea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:19.551235 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912390de-6686-4f60-a3da-94c535afaea7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:19.577114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e56c1-c937-4a61-9a98-1781461add8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:19.577790 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e56c1-c937-4a61-9a98-1781461add8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:19.578071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e56c1-c937-4a61-9a98-1781461add8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:19.578346 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e56c1-c937-4a61-9a98-1781461add8b admin admin] PATH_INFO: `/v3/users/614c7c155a814d2b93661c7feb920883` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:19.581254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e56c1-c937-4a61-9a98-1781461add8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=614c7c155a814d2b93661c7feb920883)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:19.583220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e56c1-c937-4a61-9a98-1781461add8b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:19.607599 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:19.695450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cb6e56c1-c937-4a61-9a98-1781461add8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '614c7c155a814d2b93661c7feb920883'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:19.696517 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cb6e56c1-c937-4a61-9a98-1781461add8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '614c7c155a814d2b93661c7feb920883'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:19.727592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cb6e56c1-c937-4a61-9a98-1781461add8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '614c7c155a814d2b93661c7feb920883'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:19.773684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cb6e56c1-c937-4a61-9a98-1781461add8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '614c7c155a814d2b93661c7feb920883'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:19.794935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cb6e56c1-c937-4a61-9a98-1781461add8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '614c7c155a814d2b93661c7feb920883'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:19.804138 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1127/2249] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:00:19 2022] DELETE /identity/v3/users/614c7c155a814d2b93661c7feb920883 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:00:19.809249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb95db-1aed-4273-99c3-8f8b4ccbb384 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:19.810295 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb95db-1aed-4273-99c3-8f8b4ccbb384 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:19.810742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb95db-1aed-4273-99c3-8f8b4ccbb384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:19.842087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b7fb95db-1aed-4273-99c3-8f8b4ccbb384 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:19.843893 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b7fb95db-1aed-4273-99c3-8f8b4ccbb384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:19.961214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1123/2250] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:19.967699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e56c1-c937-4a61-9a98-1781461add8b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:19.969378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cb6e56c1-c937-4a61-9a98-1781461add8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:20.095084 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e56c1-c937-4a61-9a98-1781461add8b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:20.100929 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9f2d9-5e26-4d8d-9bca-18ba783b70b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:20.101894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9f2d9-5e26-4d8d-9bca-18ba783b70b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:20.102618 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9f2d9-5e26-4d8d-9bca-18ba783b70b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:20.103222 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9f2d9-5e26-4d8d-9bca-18ba783b70b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:20.117554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9f2d9-5e26-4d8d-9bca-18ba783b70b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:20.119824 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9f2d9-5e26-4d8d-9bca-18ba783b70b6 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:20.146066 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1128/2251] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:20.880424 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb95db-1aed-4273-99c3-8f8b4ccbb384 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:20.881957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b7fb95db-1aed-4273-99c3-8f8b4ccbb384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:20.911796 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb95db-1aed-4273-99c3-8f8b4ccbb384 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:20.923694 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd078d0-4c69-4432-9bfd-cc9310f82db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:20.924485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd078d0-4c69-4432-9bfd-cc9310f82db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:20.924769 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd078d0-4c69-4432-9bfd-cc9310f82db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:20.927240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd078d0-4c69-4432-9bfd-cc9310f82db8 admin admin] PATH_INFO: `/v3/projects/804a47ef4a8a410ca25640b1986a5c39` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:20.928599 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd078d0-4c69-4432-9bfd-cc9310f82db8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=804a47ef4a8a410ca25640b1986a5c39)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:20.930541 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd078d0-4c69-4432-9bfd-cc9310f82db8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:21.058429 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3fd078d0-4c69-4432-9bfd-cc9310f82db8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '804a47ef4a8a410ca25640b1986a5c39', 'request_id': 'req-3fd078d0-4c69-4432-9bfd-cc9310f82db8'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:21.075623 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3fd078d0-4c69-4432-9bfd-cc9310f82db8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '804a47ef4a8a410ca25640b1986a5c39', 'request_id': 'req-3fd078d0-4c69-4432-9bfd-cc9310f82db8'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:21.080351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:21.086550 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3fd078d0-4c69-4432-9bfd-cc9310f82db8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '804a47ef4a8a410ca25640b1986a5c39', 'request_id': 'req-3fd078d0-4c69-4432-9bfd-cc9310f82db8'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:21.111811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3fd078d0-4c69-4432-9bfd-cc9310f82db8 admin admin] The token cache is being invalidate because project 804a47ef4a8a410ca25640b1986a5c39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:00:21.112230 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3fd078d0-4c69-4432-9bfd-cc9310f82db8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:21.114712 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3fd078d0-4c69-4432-9bfd-cc9310f82db8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:21.119003 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1124/2252] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:00:20 2022] DELETE /identity/v3/projects/804a47ef4a8a410ca25640b1986a5c39 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:00:21.124121 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa4d88-b49a-404e-848e-6d8367800527 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.124695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa4d88-b49a-404e-848e-6d8367800527 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.124695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa4d88-b49a-404e-848e-6d8367800527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:21.160207 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8dfa4d88-b49a-404e-848e-6d8367800527 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:21.162675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8dfa4d88-b49a-404e-848e-6d8367800527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:21.267088 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1129/2253] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:21.274878 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db39d40-c7bd-49aa-9d37-9039f7054948 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.275165 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db39d40-c7bd-49aa-9d37-9039f7054948 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.275441 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db39d40-c7bd-49aa-9d37-9039f7054948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:21.309360 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1db39d40-c7bd-49aa-9d37-9039f7054948 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:21.310598 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1db39d40-c7bd-49aa-9d37-9039f7054948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:21.345903 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1125/2254] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:21.348235 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa4d88-b49a-404e-848e-6d8367800527 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:21.365538 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa4d88-b49a-404e-848e-6d8367800527 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:21.387955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be06df-bd97-47d5-8d24-f902972386d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:21.389323 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be06df-bd97-47d5-8d24-f902972386d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.389684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be06df-bd97-47d5-8d24-f902972386d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.390027 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be06df-bd97-47d5-8d24-f902972386d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:21.390773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be06df-bd97-47d5-8d24-f902972386d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:00:21.391149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be06df-bd97-47d5-8d24-f902972386d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:21.393059 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be06df-bd97-47d5-8d24-f902972386d3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:21.417828 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1130/2255] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:00:21.423366 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db39d40-c7bd-49aa-9d37-9039f7054948 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:21.446484 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db39d40-c7bd-49aa-9d37-9039f7054948 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:21.453877 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ec3d7-841d-49ec-a12a-5877af2c3997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:21.455097 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ec3d7-841d-49ec-a12a-5877af2c3997 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.456395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ec3d7-841d-49ec-a12a-5877af2c3997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.457508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ec3d7-841d-49ec-a12a-5877af2c3997 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:21.458298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ec3d7-841d-49ec-a12a-5877af2c3997 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:21.460545 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ec3d7-841d-49ec-a12a-5877af2c3997 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:21.545947 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1126/2256] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:00:21 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:00:21.550651 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be06df-bd97-47d5-8d24-f902972386d3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:21.563507 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be06df-bd97-47d5-8d24-f902972386d3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:21.607966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:21.703136 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd40ba8-59b2-4328-8649-30cf66dff5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:21.704484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd40ba8-59b2-4328-8649-30cf66dff5f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.704697 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd40ba8-59b2-4328-8649-30cf66dff5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.704949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd40ba8-59b2-4328-8649-30cf66dff5f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:21.705604 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd40ba8-59b2-4328-8649-30cf66dff5f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:21.707543 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd40ba8-59b2-4328-8649-30cf66dff5f3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:21.724250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7c162-c777-4846-b233-02b64311e740 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.724884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7c162-c777-4846-b233-02b64311e740 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.725412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7c162-c777-4846-b233-02b64311e740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:21.781607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1131/2257] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:00:21 2022] POST /identity/v3/users => generated 288 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:00:21.786538 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd40ba8-59b2-4328-8649-30cf66dff5f3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:21.799109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd40ba8-59b2-4328-8649-30cf66dff5f3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:21.808913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed289442-ff8b-4f99-ae63-f7ae7c85e504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:21.812604 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed289442-ff8b-4f99-ae63-f7ae7c85e504 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.813013 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed289442-ff8b-4f99-ae63-f7ae7c85e504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.813718 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed289442-ff8b-4f99-ae63-f7ae7c85e504 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:21.814503 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed289442-ff8b-4f99-ae63-f7ae7c85e504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:21.818675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed289442-ff8b-4f99-ae63-f7ae7c85e504 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:21.824738 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-6de7c162-c777-4846-b233-02b64311e740 None None] MFA Rules not processed for user `bf53ca52f58d4170a256aae15fdaf8e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:21.826160 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6de7c162-c777-4846-b233-02b64311e740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:21.831342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1132/2258] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:00:21.835879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed289442-ff8b-4f99-ae63-f7ae7c85e504 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:21.857821 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed289442-ff8b-4f99-ae63-f7ae7c85e504 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:21.871949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c442e-d00c-47f4-9a08-4bddf5d166ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:21.874650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c442e-d00c-47f4-9a08-4bddf5d166ca admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.875120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c442e-d00c-47f4-9a08-4bddf5d166ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.875575 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c442e-d00c-47f4-9a08-4bddf5d166ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:21.878132 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c442e-d00c-47f4-9a08-4bddf5d166ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:21.880700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c442e-d00c-47f4-9a08-4bddf5d166ca admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:21.902659 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1133/2259] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:00:21.907770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c442e-d00c-47f4-9a08-4bddf5d166ca admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:21.924944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c442e-d00c-47f4-9a08-4bddf5d166ca admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:21.930768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e05911-3f1d-4c44-9991-4e34fe39879b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:21.931748 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e05911-3f1d-4c44-9991-4e34fe39879b admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.932207 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e05911-3f1d-4c44-9991-4e34fe39879b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.932594 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e05911-3f1d-4c44-9991-4e34fe39879b admin admin] PATH_INFO: `/v3/projects/6e585066a2274ef3b5a3ff29933e42ab/users/d5a1c7fe370643b08d6775190a155d86/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:21.946002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1127/2260] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:00:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:21.955366 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e05911-3f1d-4c44-9991-4e34fe39879b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e585066a2274ef3b5a3ff29933e42ab, user_id=d5a1c7fe370643b08d6775190a155d86, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:21.956981 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7c162-c777-4846-b233-02b64311e740 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:21.958598 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6de7c162-c777-4846-b233-02b64311e740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:21.959011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e05911-3f1d-4c44-9991-4e34fe39879b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:21.981786 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1134/2261] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:00:21 2022] PUT /identity/v3/projects/6e585066a2274ef3b5a3ff29933e42ab/users/d5a1c7fe370643b08d6775190a155d86/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:00:21.996398 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d7ff0-4935-40cc-9d46-355a34ea4b1a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:21.996899 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d7ff0-4935-40cc-9d46-355a34ea4b1a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:21.997291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d7ff0-4935-40cc-9d46-355a34ea4b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:22.048842 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-119d7ff0-4935-40cc-9d46-355a34ea4b1a None None] MFA Rules not processed for user `d5a1c7fe370643b08d6775190a155d86`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:22.050284 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-119d7ff0-4935-40cc-9d46-355a34ea4b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:22.123269 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7c162-c777-4846-b233-02b64311e740 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:22.217989 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1135/2262] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:00:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:22.220379 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d7ff0-4935-40cc-9d46-355a34ea4b1a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:22.230495 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:22.232555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-119d7ff0-4935-40cc-9d46-355a34ea4b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:22.306608 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20546f2-48fd-424c-aa1a-8002e93e1e0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:22.307241 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20546f2-48fd-424c-aa1a-8002e93e1e0d service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:22.307524 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20546f2-48fd-424c-aa1a-8002e93e1e0d service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:22.307796 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20546f2-48fd-424c-aa1a-8002e93e1e0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:22.312212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b20546f2-48fd-424c-aa1a-8002e93e1e0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:22.391711 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d7ff0-4935-40cc-9d46-355a34ea4b1a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:22.396223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e411284-172f-4020-9d82-551b7523270b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:22.397227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e411284-172f-4020-9d82-551b7523270b service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:22.397227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e411284-172f-4020-9d82-551b7523270b service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:22.397469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e411284-172f-4020-9d82-551b7523270b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:22.403358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:22.413001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e411284-172f-4020-9d82-551b7523270b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:22.414615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e411284-172f-4020-9d82-551b7523270b service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:22.440188 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1136/2263] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:22.452696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20546f2-48fd-424c-aa1a-8002e93e1e0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:22.454682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20546f2-48fd-424c-aa1a-8002e93e1e0d service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:22.467647 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1128/2264] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:00:24.074299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938fd75-2312-4d54-99f9-71eab6d6573c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:24.074804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938fd75-2312-4d54-99f9-71eab6d6573c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:24.074804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938fd75-2312-4d54-99f9-71eab6d6573c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:24.172049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7938fd75-2312-4d54-99f9-71eab6d6573c None None] MFA Rules not processed for user `d5a1c7fe370643b08d6775190a155d86`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:24.176835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7938fd75-2312-4d54-99f9-71eab6d6573c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:24.202810 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b93fa-ef75-4127-99e3-66e88ba6c164 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:24.203302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b93fa-ef75-4127-99e3-66e88ba6c164 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:24.203414 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b93fa-ef75-4127-99e3-66e88ba6c164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:24.220839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1137/2265] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:00:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:24.273379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-300b93fa-ef75-4127-99e3-66e88ba6c164 None None] MFA Rules not processed for user `edc56fc598e741ceab98edd3b9afa28c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:24.277215 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-300b93fa-ef75-4127-99e3-66e88ba6c164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:24.307162 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1129/2266] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:00:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:28.471591 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f69ec-363c-4e35-b123-e4ea4026a724 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:28.472300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f69ec-363c-4e35-b123-e4ea4026a724 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:28.472300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f69ec-363c-4e35-b123-e4ea4026a724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:28.543126 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7f5f69ec-363c-4e35-b123-e4ea4026a724 None None] MFA Rules not processed for user `bd0929f371e4442eb4eb6e12a96bbfc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:28.549882 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7f5f69ec-363c-4e35-b123-e4ea4026a724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:28.732030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1138/2267] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:00:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:29.367196 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b93fa-ef75-4127-99e3-66e88ba6c164 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:29.369484 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-300b93fa-ef75-4127-99e3-66e88ba6c164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:29.469987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b93fa-ef75-4127-99e3-66e88ba6c164 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:29.475481 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7095e7-7edd-4ff4-aa37-bce522da903d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:29.480191 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7095e7-7edd-4ff4-aa37-bce522da903d service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:29.481324 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7095e7-7edd-4ff4-aa37-bce522da903d service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:29.481763 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7095e7-7edd-4ff4-aa37-bce522da903d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:29.483876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9a7095e7-7edd-4ff4-aa37-bce522da903d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:29.500222 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7095e7-7edd-4ff4-aa37-bce522da903d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:29.502566 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7095e7-7edd-4ff4-aa37-bce522da903d service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:29.516874 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1130/2268] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:32.378743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f69ec-363c-4e35-b123-e4ea4026a724 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:32.380220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7f5f69ec-363c-4e35-b123-e4ea4026a724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:32.446769 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f69ec-363c-4e35-b123-e4ea4026a724 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:32.450729 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6759bc-cc54-4517-bc02-b8e26b63f3e2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5979387f7aa54619b8d429368c306cc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:32.451341 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6759bc-cc54-4517-bc02-b8e26b63f3e2 service glance] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:32.451621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6759bc-cc54-4517-bc02-b8e26b63f3e2 service glance] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:32.451891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6759bc-cc54-4517-bc02-b8e26b63f3e2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:32.453788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ab6759bc-cc54-4517-bc02-b8e26b63f3e2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:32.467382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6759bc-cc54-4517-bc02-b8e26b63f3e2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:32.468839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6759bc-cc54-4517-bc02-b8e26b63f3e2 service glance] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:32.479658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1139/2269] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:39.510953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=66 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:42.475012 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=57 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:44.318189 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f58ca-7203-462d-a347-5b4916a41bea None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:44.319215 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f58ca-7203-462d-a347-5b4916a41bea None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:44.319215 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f58ca-7203-462d-a347-5b4916a41bea None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:44.320470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1131/2270] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 24 09:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:00:44.329581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef228ec-14a1-4eff-848a-3a15dea277d1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:44.330330 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef228ec-14a1-4eff-848a-3a15dea277d1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:44.332080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef228ec-14a1-4eff-848a-3a15dea277d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:44.365439 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-eef228ec-14a1-4eff-848a-3a15dea277d1 None None] MFA Rules not processed for user `ca56dd7216fb4946946b2801e144fec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:44.366805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-eef228ec-14a1-4eff-848a-3a15dea277d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:44.379722 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1140/2271] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 24 09:00:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:44.393726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f58ca-7203-462d-a347-5b4916a41bea None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:44.406824 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f58ca-7203-462d-a347-5b4916a41bea None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:44.411049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd352379-5bf0-4ed8-b298-ec20cc443939 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5979387f7aa54619b8d429368c306cc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:44.411952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd352379-5bf0-4ed8-b298-ec20cc443939 service glance] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:44.412334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd352379-5bf0-4ed8-b298-ec20cc443939 service glance] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:44.412622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd352379-5bf0-4ed8-b298-ec20cc443939 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:44.428737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd352379-5bf0-4ed8-b298-ec20cc443939 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:44.430300 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd352379-5bf0-4ed8-b298-ec20cc443939 service glance] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:44.444149 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1132/2272] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:00:44.542032 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56265b3-ba57-4da1-94c8-dfd8edcfad9a None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:44.542420 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56265b3-ba57-4da1-94c8-dfd8edcfad9a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:44.542708 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56265b3-ba57-4da1-94c8-dfd8edcfad9a None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:44.544999 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1141/2273] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 24 09:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:00:44.556461 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bca1d-72eb-4e98-bf80-d928f081f07e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:44.556461 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bca1d-72eb-4e98-bf80-d928f081f07e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:44.556461 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bca1d-72eb-4e98-bf80-d928f081f07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:44.587522 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-485bca1d-72eb-4e98-bf80-d928f081f07e None None] MFA Rules not processed for user `2ee2e93901914b1d82607a3ae0e2acf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:44.588926 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-485bca1d-72eb-4e98-bf80-d928f081f07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:44.597561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1133/2274] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 24 09:00:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:44.614817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56265b3-ba57-4da1-94c8-dfd8edcfad9a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:44.617433 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a56265b3-ba57-4da1-94c8-dfd8edcfad9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:44.633745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56265b3-ba57-4da1-94c8-dfd8edcfad9a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:44.637770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540c0bb-cd38-48eb-89f1-d8971c65a6c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:44.638386 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540c0bb-cd38-48eb-89f1-d8971c65a6c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:44.638667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540c0bb-cd38-48eb-89f1-d8971c65a6c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:44.638938 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540c0bb-cd38-48eb-89f1-d8971c65a6c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:44.648539 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540c0bb-cd38-48eb-89f1-d8971c65a6c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:44.650006 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540c0bb-cd38-48eb-89f1-d8971c65a6c9 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:44.659698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1142/2275] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:44.766243 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d668a78-1749-42af-98d2-43d356b6512b None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:44.766552 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d668a78-1749-42af-98d2-43d356b6512b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:44.766840 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d668a78-1749-42af-98d2-43d356b6512b None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:44.768492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1134/2276] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 24 09:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:00:44.776253 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34889fb2-4275-4f83-9c0d-8271ee7e4b3a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:44.776887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34889fb2-4275-4f83-9c0d-8271ee7e4b3a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:44.777408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34889fb2-4275-4f83-9c0d-8271ee7e4b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:44.802161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-34889fb2-4275-4f83-9c0d-8271ee7e4b3a None None] MFA Rules not processed for user `ca56dd7216fb4946946b2801e144fec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:44.803396 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-34889fb2-4275-4f83-9c0d-8271ee7e4b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:44.816732 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1143/2277] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 24 09:00:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:00:44.832217 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d668a78-1749-42af-98d2-43d356b6512b None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:44.850695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d668a78-1749-42af-98d2-43d356b6512b None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:44.857047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fe2ca-2c4b-4b80-b527-5ee9795d1cb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:44.858110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fe2ca-2c4b-4b80-b527-5ee9795d1cb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:44.858610 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fe2ca-2c4b-4b80-b527-5ee9795d1cb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:44.859112 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fe2ca-2c4b-4b80-b527-5ee9795d1cb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:44.878014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633fe2ca-2c4b-4b80-b527-5ee9795d1cb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:44.879861 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633fe2ca-2c4b-4b80-b527-5ee9795d1cb5 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:44.894162 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1135/2278] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:45.609290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34889fb2-4275-4f83-9c0d-8271ee7e4b3a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:45.622252 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34889fb2-4275-4f83-9c0d-8271ee7e4b3a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:45.630947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052aaaf4-d5b6-4a28-b7ef-940da3a26bf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:45.631573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052aaaf4-d5b6-4a28-b7ef-940da3a26bf6 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:45.632014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052aaaf4-d5b6-4a28-b7ef-940da3a26bf6 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:45.632498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052aaaf4-d5b6-4a28-b7ef-940da3a26bf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:45.634581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-052aaaf4-d5b6-4a28-b7ef-940da3a26bf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:45.713355 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052aaaf4-d5b6-4a28-b7ef-940da3a26bf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:45.715502 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052aaaf4-d5b6-4a28-b7ef-940da3a26bf6 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:45.728290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1144/2279] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:45 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:50.667382 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fe2ca-2c4b-4b80-b527-5ee9795d1cb5 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:50.676932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fe2ca-2c4b-4b80-b527-5ee9795d1cb5 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:50.680839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1b471-46c9-422a-8d5b-c5962590f9e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:50.681451 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1b471-46c9-422a-8d5b-c5962590f9e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:50.681729 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1b471-46c9-422a-8d5b-c5962590f9e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:50.681999 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1b471-46c9-422a-8d5b-c5962590f9e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:50.683787 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-afb1b471-46c9-422a-8d5b-c5962590f9e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:50.694832 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1b471-46c9-422a-8d5b-c5962590f9e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:50.696306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1b471-46c9-422a-8d5b-c5962590f9e1 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:50.704752 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1136/2280] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:54.022136 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de01326-c039-4207-95a2-6978cc8fac67 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:54.022737 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de01326-c039-4207-95a2-6978cc8fac67 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:54.022737 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de01326-c039-4207-95a2-6978cc8fac67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:54.043842 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9de01326-c039-4207-95a2-6978cc8fac67 None None] MFA Rules not processed for user `b11abed8d4424a1e9957c3dc9aefadd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:54.045112 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9de01326-c039-4207-95a2-6978cc8fac67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:54.111460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1145/2281] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 24 09:00:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:54.125043 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1b471-46c9-422a-8d5b-c5962590f9e1 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:54.126873 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-afb1b471-46c9-422a-8d5b-c5962590f9e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:54.209003 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1b471-46c9-422a-8d5b-c5962590f9e1 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:54.214455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fb4d6-ea91-4899-a006-3db69e817c82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46b1417d4ac24c8982b2052f4e99042a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:54.215085 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fb4d6-ea91-4899-a006-3db69e817c82 service swift] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:54.215365 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fb4d6-ea91-4899-a006-3db69e817c82 service swift] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:54.215638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fb4d6-ea91-4899-a006-3db69e817c82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:54.224634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fb4d6-ea91-4899-a006-3db69e817c82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:54.226318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fb4d6-ea91-4899-a006-3db69e817c82 service swift] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:54.235303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1137/2282] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 24 09:00:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:00:54.885689 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdd76c-48cb-45e2-995f-67a24430deb3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:54.886785 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdd76c-48cb-45e2-995f-67a24430deb3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:54.886930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdd76c-48cb-45e2-995f-67a24430deb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:54.910009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e809d-7177-4f52-8a32-6a9c92436589 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:54.910278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e809d-7177-4f52-8a32-6a9c92436589 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:54.910568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e809d-7177-4f52-8a32-6a9c92436589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:54.934863 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e97e809d-7177-4f52-8a32-6a9c92436589 None None] MFA Rules not processed for user `b11abed8d4424a1e9957c3dc9aefadd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:54.936299 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e97e809d-7177-4f52-8a32-6a9c92436589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:54.941542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1dcdd76c-48cb-45e2-995f-67a24430deb3 None None] MFA Rules not processed for user `b11abed8d4424a1e9957c3dc9aefadd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:54.944288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1dcdd76c-48cb-45e2-995f-67a24430deb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:54.952859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1138/2283] 10.222.0.67 () {58 vars in 967 bytes} [Wed Aug 24 09:00:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:00:54.963026 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1146/2284] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 24 09:00:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:54.966533 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e809d-7177-4f52-8a32-6a9c92436589 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:54.979555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdd76c-48cb-45e2-995f-67a24430deb3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:54.980535 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e809d-7177-4f52-8a32-6a9c92436589 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:54.986263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4bbd1-2c1a-474a-9460-56b993559ce7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46b1417d4ac24c8982b2052f4e99042a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:54.988723 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4bbd1-2c1a-474a-9460-56b993559ce7 service swift] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:54.989043 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4bbd1-2c1a-474a-9460-56b993559ce7 service swift] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:54.989317 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4bbd1-2c1a-474a-9460-56b993559ce7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:54.994290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdd76c-48cb-45e2-995f-67a24430deb3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:54.998933 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4bbd1-2c1a-474a-9460-56b993559ce7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:55.000319 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2accbdaa-8f06-444a-ab90-cfd93fa994a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46b1417d4ac24c8982b2052f4e99042a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:55.000585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4bbd1-2c1a-474a-9460-56b993559ce7 service swift] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:55.001070 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accbdaa-8f06-444a-ab90-cfd93fa994a2 service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:55.001417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accbdaa-8f06-444a-ab90-cfd93fa994a2 service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:55.001749 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accbdaa-8f06-444a-ab90-cfd93fa994a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:55.012612 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1139/2285] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 24 09:00:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:00:55.019566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2accbdaa-8f06-444a-ab90-cfd93fa994a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:55.021303 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2accbdaa-8f06-444a-ab90-cfd93fa994a2 service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:55.033781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1147/2286] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 24 09:00:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 09:00:55.358744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e990f96-99e9-46a6-826c-d1b5def5e218 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:55.359277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e990f96-99e9-46a6-826c-d1b5def5e218 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:55.359277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e990f96-99e9-46a6-826c-d1b5def5e218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:55.379781 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-6e990f96-99e9-46a6-826c-d1b5def5e218 None None] MFA Rules not processed for user `b11abed8d4424a1e9957c3dc9aefadd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:55.381023 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6e990f96-99e9-46a6-826c-d1b5def5e218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:55.390029 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1140/2287] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 24 09:00:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:55.405574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2accbdaa-8f06-444a-ab90-cfd93fa994a2 service swift] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:55.418531 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2accbdaa-8f06-444a-ab90-cfd93fa994a2 service swift] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:55.425332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f5399-01af-4e1b-a328-5044c6b6798d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46b1417d4ac24c8982b2052f4e99042a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:55.426561 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f5399-01af-4e1b-a328-5044c6b6798d service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:55.426875 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f5399-01af-4e1b-a328-5044c6b6798d service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:55.428195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f5399-01af-4e1b-a328-5044c6b6798d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:55.440975 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f5399-01af-4e1b-a328-5044c6b6798d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:55.442601 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f5399-01af-4e1b-a328-5044c6b6798d service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:55.453780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1148/2288] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 24 09:00:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:00:55.982672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8e971-2d73-4b31-bba7-021c50313923 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:55.983302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8e971-2d73-4b31-bba7-021c50313923 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:55.983836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8e971-2d73-4b31-bba7-021c50313923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:56.009185 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-42f8e971-2d73-4b31-bba7-021c50313923 None None] MFA Rules not processed for user `b11abed8d4424a1e9957c3dc9aefadd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:56.010909 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-42f8e971-2d73-4b31-bba7-021c50313923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:56.026902 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1141/2289] 10.222.0.67 () {58 vars in 967 bytes} [Wed Aug 24 09:00:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:56.048912 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f5399-01af-4e1b-a328-5044c6b6798d service swift] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:56.067302 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f5399-01af-4e1b-a328-5044c6b6798d service swift] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:56.075981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c019d1-57cd-4174-ac6e-1ad43f3902f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46b1417d4ac24c8982b2052f4e99042a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:56.076670 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c019d1-57cd-4174-ac6e-1ad43f3902f9 service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:56.076949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c019d1-57cd-4174-ac6e-1ad43f3902f9 service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:56.077406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c019d1-57cd-4174-ac6e-1ad43f3902f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:56.094065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c019d1-57cd-4174-ac6e-1ad43f3902f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:56.096135 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c019d1-57cd-4174-ac6e-1ad43f3902f9 service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:56.108378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1149/2290] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 24 09:00:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:00:57.455379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8e971-2d73-4b31-bba7-021c50313923 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:57.457253 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-42f8e971-2d73-4b31-bba7-021c50313923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:57.505603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8e971-2d73-4b31-bba7-021c50313923 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:57.528049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56968be0-6c19-47d3-9c27-29a0c81719ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:57.528723 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56968be0-6c19-47d3-9c27-29a0c81719ad service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:57.529111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56968be0-6c19-47d3-9c27-29a0c81719ad service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:57.529689 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56968be0-6c19-47d3-9c27-29a0c81719ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:57.532305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-56968be0-6c19-47d3-9c27-29a0c81719ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:57.551486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56968be0-6c19-47d3-9c27-29a0c81719ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:57.553429 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56968be0-6c19-47d3-9c27-29a0c81719ad service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:57.568551 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1142/2291] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:58.705203 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c019d1-57cd-4174-ac6e-1ad43f3902f9 service swift] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:58.754994 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:58.791800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c019d1-57cd-4174-ac6e-1ad43f3902f9 service swift] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:58.799454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ccf629-a3fb-4299-a7fb-714607e22d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:58.800243 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccf629-a3fb-4299-a7fb-714607e22d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:58.800685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccf629-a3fb-4299-a7fb-714607e22d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:58.800989 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccf629-a3fb-4299-a7fb-714607e22d43 admin admin] PATH_INFO: `/v3/users/d5a1c7fe370643b08d6775190a155d86` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:58.802175 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ccf629-a3fb-4299-a7fb-714607e22d43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5a1c7fe370643b08d6775190a155d86)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:58.804339 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ccf629-a3fb-4299-a7fb-714607e22d43 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:58.874519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-50ccf629-a3fb-4299-a7fb-714607e22d43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5a1c7fe370643b08d6775190a155d86'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:58.875568 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-50ccf629-a3fb-4299-a7fb-714607e22d43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5a1c7fe370643b08d6775190a155d86'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:58.888068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-50ccf629-a3fb-4299-a7fb-714607e22d43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5a1c7fe370643b08d6775190a155d86'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:58.897871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-50ccf629-a3fb-4299-a7fb-714607e22d43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5a1c7fe370643b08d6775190a155d86'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:58.913337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-50ccf629-a3fb-4299-a7fb-714607e22d43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5a1c7fe370643b08d6775190a155d86'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:58.928407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1150/2292] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:00:58 2022] DELETE /identity/v3/users/d5a1c7fe370643b08d6775190a155d86 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:00:58.930282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660e1b3-6bcd-4f05-b11c-5d9099585742 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:58.931399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660e1b3-6bcd-4f05-b11c-5d9099585742 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:58.931682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660e1b3-6bcd-4f05-b11c-5d9099585742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:58.963599 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7660e1b3-6bcd-4f05-b11c-5d9099585742 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:00:58.965192 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7660e1b3-6bcd-4f05-b11c-5d9099585742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:59.028721 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:00:59.039577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1143/2293] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:00:59.049248 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ccf629-a3fb-4299-a7fb-714607e22d43 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:59.051051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-50ccf629-a3fb-4299-a7fb-714607e22d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:59.118190 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ccf629-a3fb-4299-a7fb-714607e22d43 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:59.123521 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c568e9-d948-4d24-a228-dc1e8a4bcb3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:59.124313 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c568e9-d948-4d24-a228-dc1e8a4bcb3f service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:59.124713 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c568e9-d948-4d24-a228-dc1e8a4bcb3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:59.125082 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c568e9-d948-4d24-a228-dc1e8a4bcb3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:59.137392 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c568e9-d948-4d24-a228-dc1e8a4bcb3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:59.138813 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c568e9-d948-4d24-a228-dc1e8a4bcb3f service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:59.150707 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1151/2294] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:00:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:00:59.339347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660e1b3-6bcd-4f05-b11c-5d9099585742 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:00:59.341431 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7660e1b3-6bcd-4f05-b11c-5d9099585742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:00:59.386777 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660e1b3-6bcd-4f05-b11c-5d9099585742 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:00:59.393081 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ca31c-9121-471a-83fc-a4acee44181f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:00:59.394164 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ca31c-9121-471a-83fc-a4acee44181f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:00:59.394678 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ca31c-9121-471a-83fc-a4acee44181f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:00:59.395159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ca31c-9121-471a-83fc-a4acee44181f admin admin] PATH_INFO: `/v3/projects/6e585066a2274ef3b5a3ff29933e42ab` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:00:59.396722 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ca31c-9121-471a-83fc-a4acee44181f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e585066a2274ef3b5a3ff29933e42ab)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:00:59.399084 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ca31c-9121-471a-83fc-a4acee44181f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:00:59.463482 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-391ca31c-9121-471a-83fc-a4acee44181f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e585066a2274ef3b5a3ff29933e42ab', 'request_id': 'req-391ca31c-9121-471a-83fc-a4acee44181f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:59.472820 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-391ca31c-9121-471a-83fc-a4acee44181f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e585066a2274ef3b5a3ff29933e42ab', 'request_id': 'req-391ca31c-9121-471a-83fc-a4acee44181f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:59.479900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-391ca31c-9121-471a-83fc-a4acee44181f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e585066a2274ef3b5a3ff29933e42ab', 'request_id': 'req-391ca31c-9121-471a-83fc-a4acee44181f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:59.493513 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-391ca31c-9121-471a-83fc-a4acee44181f admin admin] The token cache is being invalidate because project 6e585066a2274ef3b5a3ff29933e42ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:00:59.494078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-391ca31c-9121-471a-83fc-a4acee44181f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:59.494938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-391ca31c-9121-471a-83fc-a4acee44181f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:00:59.505776 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1144/2295] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:00:59 2022] DELETE /identity/v3/projects/6e585066a2274ef3b5a3ff29933e42ab => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:01:08.385493 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c568e9-d948-4d24-a228-dc1e8a4bcb3f service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:08.387237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-85c568e9-d948-4d24-a228-dc1e8a4bcb3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:08.399985 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c568e9-d948-4d24-a228-dc1e8a4bcb3f service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:08.451312 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc424b0-7913-4edb-8e75-8351e0d91215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:08.451915 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc424b0-7913-4edb-8e75-8351e0d91215 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:08.452217 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc424b0-7913-4edb-8e75-8351e0d91215 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:08.452482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc424b0-7913-4edb-8e75-8351e0d91215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:08.454388 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6fc424b0-7913-4edb-8e75-8351e0d91215 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:08.461793 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:08.464756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc424b0-7913-4edb-8e75-8351e0d91215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:08.466665 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc424b0-7913-4edb-8e75-8351e0d91215 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:08.476931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1152/2296] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:09.489709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:18.477754 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:25.089577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96265c96-f36d-4921-b68b-956af173b077 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:25.090261 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96265c96-f36d-4921-b68b-956af173b077 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:25.090444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96265c96-f36d-4921-b68b-956af173b077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:25.114529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-96265c96-f36d-4921-b68b-956af173b077 None None] MFA Rules not processed for user `b11abed8d4424a1e9957c3dc9aefadd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:01:25.115878 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-96265c96-f36d-4921-b68b-956af173b077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:25.176530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1145/2297] 10.222.0.67 () {58 vars in 967 bytes} [Wed Aug 24 09:01:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:01:25.200507 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc424b0-7913-4edb-8e75-8351e0d91215 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:25.201955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6fc424b0-7913-4edb-8e75-8351e0d91215 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:25.285417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc424b0-7913-4edb-8e75-8351e0d91215 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:25.290696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e2d0f-7f97-431a-8c71-3a83aa801481 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46b1417d4ac24c8982b2052f4e99042a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:25.291325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e2d0f-7f97-431a-8c71-3a83aa801481 service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:25.291732 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e2d0f-7f97-431a-8c71-3a83aa801481 service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:25.292312 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e2d0f-7f97-431a-8c71-3a83aa801481 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:25.302447 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e2d0f-7f97-431a-8c71-3a83aa801481 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:25.303985 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e2d0f-7f97-431a-8c71-3a83aa801481 service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:25.313689 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1153/2298] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 24 09:01:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:01:28.128009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96265c96-f36d-4921-b68b-956af173b077 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:28.130326 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-96265c96-f36d-4921-b68b-956af173b077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:28.142838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96265c96-f36d-4921-b68b-956af173b077 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:28.146745 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f35746-b5e1-44c8-994c-7b9cd836fe92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:28.147365 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f35746-b5e1-44c8-994c-7b9cd836fe92 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:28.147685 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f35746-b5e1-44c8-994c-7b9cd836fe92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:28.147958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f35746-b5e1-44c8-994c-7b9cd836fe92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:28.149766 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-08f35746-b5e1-44c8-994c-7b9cd836fe92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:28.220405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f35746-b5e1-44c8-994c-7b9cd836fe92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:28.221860 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f35746-b5e1-44c8-994c-7b9cd836fe92 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:28.232302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1146/2299] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:29.860787 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e2d0f-7f97-431a-8c71-3a83aa801481 service swift] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:29.862203 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e02e2d0f-7f97-431a-8c71-3a83aa801481 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:29.959423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e2d0f-7f97-431a-8c71-3a83aa801481 service swift] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:29.970650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15a64d-9d81-4dc6-9489-86d5bd0ddada service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:29.971723 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15a64d-9d81-4dc6-9489-86d5bd0ddada service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:29.972262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15a64d-9d81-4dc6-9489-86d5bd0ddada service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:29.972752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15a64d-9d81-4dc6-9489-86d5bd0ddada service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:29.975602 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ac15a64d-9d81-4dc6-9489-86d5bd0ddada service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:29.997819 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15a64d-9d81-4dc6-9489-86d5bd0ddada service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:30.000952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15a64d-9d81-4dc6-9489-86d5bd0ddada service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:30.021800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1154/2300] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:35.869759 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f35746-b5e1-44c8-994c-7b9cd836fe92 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:35.871195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-08f35746-b5e1-44c8-994c-7b9cd836fe92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:35.886992 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f35746-b5e1-44c8-994c-7b9cd836fe92 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:35.892750 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760032c-5dee-4290-bbb3-9468c059160e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:35.893306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760032c-5dee-4290-bbb3-9468c059160e service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:35.893582 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760032c-5dee-4290-bbb3-9468c059160e service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:35.893848 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760032c-5dee-4290-bbb3-9468c059160e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:35.896307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f760032c-5dee-4290-bbb3-9468c059160e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:35.955545 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760032c-5dee-4290-bbb3-9468c059160e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:35.957225 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760032c-5dee-4290-bbb3-9468c059160e service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:35.968124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1147/2301] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:35 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:36.510701 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15a64d-9d81-4dc6-9489-86d5bd0ddada service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:01:36.522038 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15a64d-9d81-4dc6-9489-86d5bd0ddada service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:01:36.527838 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732b6ed-9523-45ce-8e97-ccb248c57091 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:01:36.528592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732b6ed-9523-45ce-8e97-ccb248c57091 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:01:36.528869 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732b6ed-9523-45ce-8e97-ccb248c57091 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:01:36.529143 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732b6ed-9523-45ce-8e97-ccb248c57091 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:01:36.530890 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9732b6ed-9523-45ce-8e97-ccb248c57091 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:01:36.590538 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732b6ed-9523-45ce-8e97-ccb248c57091 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:01:36.592161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732b6ed-9523-45ce-8e97-ccb248c57091 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:01:36.603377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1155/2302] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:01:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:01:45.963425 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=83 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:01:46.597175 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=94 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:05.606626 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce790ee3-e213-49e6-83da-452e3e4d28e9 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:05.607225 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce790ee3-e213-49e6-83da-452e3e4d28e9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:05.607225 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce790ee3-e213-49e6-83da-452e3e4d28e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:05.627117 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-ce790ee3-e213-49e6-83da-452e3e4d28e9 None None] MFA Rules not processed for user `8ee5364270094596b64b08a8ac56f6ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:02:05.628336 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ce790ee3-e213-49e6-83da-452e3e4d28e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:05.635777 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1148/2303] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:02:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:02:15.621562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:44.035457 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc722bc1-b8f3-4d6c-947a-24a83b7f8fe3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:44.036005 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc722bc1-b8f3-4d6c-947a-24a83b7f8fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:44.036005 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc722bc1-b8f3-4d6c-947a-24a83b7f8fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:44.056004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-fc722bc1-b8f3-4d6c-947a-24a83b7f8fe3 None None] MFA Rules not processed for user `bd0929f371e4442eb4eb6e12a96bbfc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:02:44.057692 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fc722bc1-b8f3-4d6c-947a-24a83b7f8fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:44.113013 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1156/2304] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:02:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:02:44.753720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce790ee3-e213-49e6-83da-452e3e4d28e9 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:44.755209 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ce790ee3-e213-49e6-83da-452e3e4d28e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:44.835963 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce790ee3-e213-49e6-83da-452e3e4d28e9 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:44.841243 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a041db-0eda-4d97-a762-91956b35aabf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33a0844a1c8f4ea4932e12d53e5ee9ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:44.841918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a041db-0eda-4d97-a762-91956b35aabf service placement] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:44.842360 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a041db-0eda-4d97-a762-91956b35aabf service placement] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:44.842792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a041db-0eda-4d97-a762-91956b35aabf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:44.844765 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d7a041db-0eda-4d97-a762-91956b35aabf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:44.856820 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a041db-0eda-4d97-a762-91956b35aabf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:44.858989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a041db-0eda-4d97-a762-91956b35aabf service placement] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:44.868881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1149/2305] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:02:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:02:52.254468 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33571f02-feba-46e6-b8af-fed3c341fabf None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:52.254979 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33571f02-feba-46e6-b8af-fed3c341fabf None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:52.255100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33571f02-feba-46e6-b8af-fed3c341fabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:52.274115 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-33571f02-feba-46e6-b8af-fed3c341fabf None None] MFA Rules not processed for user `f0d0eb9dc0f743c3985218901334b336`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:02:52.275315 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-33571f02-feba-46e6-b8af-fed3c341fabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:52.331823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1157/2306] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:02:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:02:54.864682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=32 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:02:56.709651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a041db-0eda-4d97-a762-91956b35aabf service placement] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:02:56.724420 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a041db-0eda-4d97-a762-91956b35aabf service placement] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:02:56.731743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e29ff-bcb0-4567-a82d-43c6eb9875fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:02:56.732635 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e29ff-bcb0-4567-a82d-43c6eb9875fc service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:02:56.733658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e29ff-bcb0-4567-a82d-43c6eb9875fc service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:02:56.734130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e29ff-bcb0-4567-a82d-43c6eb9875fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:02:56.736055 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b62e29ff-bcb0-4567-a82d-43c6eb9875fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:02:56.755017 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e29ff-bcb0-4567-a82d-43c6eb9875fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:02:56.756964 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e29ff-bcb0-4567-a82d-43c6eb9875fc service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:02:56.776462 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1150/2307] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:02:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:02.324727 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=50 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:06.146433 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6effddd-6ebf-43d4-aa92-381ec01784f6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:06.147292 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6effddd-6ebf-43d4-aa92-381ec01784f6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:06.147292 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6effddd-6ebf-43d4-aa92-381ec01784f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:06.168554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a6effddd-6ebf-43d4-aa92-381ec01784f6 None None] MFA Rules not processed for user `46b3aa99a5e84ef2a8df193628feb0d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:06.169753 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a6effddd-6ebf-43d4-aa92-381ec01784f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:06.228374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1158/2308] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:06.746417 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e29ff-bcb0-4567-a82d-43c6eb9875fc service nova] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:06.757009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e29ff-bcb0-4567-a82d-43c6eb9875fc service nova] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:06.761022 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10394ce3-6db8-40b6-b212-0f779828ba46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:06.761632 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10394ce3-6db8-40b6-b212-0f779828ba46 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:06.762227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10394ce3-6db8-40b6-b212-0f779828ba46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:06.762227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10394ce3-6db8-40b6-b212-0f779828ba46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:06.763890 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-10394ce3-6db8-40b6-b212-0f779828ba46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:06.779338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10394ce3-6db8-40b6-b212-0f779828ba46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:06.781585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10394ce3-6db8-40b6-b212-0f779828ba46 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:06.791602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1151/2309] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:16.221543 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=25 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:16.787962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=12 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:23.915193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410569f-c482-4c64-bead-c503a415567e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:23.915779 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410569f-c482-4c64-bead-c503a415567e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:23.915779 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410569f-c482-4c64-bead-c503a415567e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:23.935446 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a410569f-c482-4c64-bead-c503a415567e None None] MFA Rules not processed for user `f0dc2dcc5327489c9ecc28a43ab7c0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:23.936675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a410569f-c482-4c64-bead-c503a415567e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:23.943480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1159/2310] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:33.929052 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:41.927580 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10394ce3-6db8-40b6-b212-0f779828ba46 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:41.929268 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-10394ce3-6db8-40b6-b212-0f779828ba46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:41.975438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10394ce3-6db8-40b6-b212-0f779828ba46 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:41.978937 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ef19f-e544-45ae-b4da-20c78693fec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:41.979549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ef19f-e544-45ae-b4da-20c78693fec3 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:41.979831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ef19f-e544-45ae-b4da-20c78693fec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:41.980149 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ef19f-e544-45ae-b4da-20c78693fec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:41.981881 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5f4ef19f-e544-45ae-b4da-20c78693fec3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:42.112073 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ef19f-e544-45ae-b4da-20c78693fec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:42.113780 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ef19f-e544-45ae-b4da-20c78693fec3 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:42.134743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1152/2311] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:45.841766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410569f-c482-4c64-bead-c503a415567e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:45.843232 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a410569f-c482-4c64-bead-c503a415567e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:45.888806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410569f-c482-4c64-bead-c503a415567e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:45.893481 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7f6df-fa9d-4822-9059-596241272922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:45.894107 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7f6df-fa9d-4822-9059-596241272922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:45.894384 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7f6df-fa9d-4822-9059-596241272922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:45.894655 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7f6df-fa9d-4822-9059-596241272922 admin admin] PATH_INFO: `/v3/users/8ee5364270094596b64b08a8ac56f6ce` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:45.895787 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7f6df-fa9d-4822-9059-596241272922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ee5364270094596b64b08a8ac56f6ce)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:45.897728 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7f6df-fa9d-4822-9059-596241272922 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:45.943271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-32e7f6df-fa9d-4822-9059-596241272922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ee5364270094596b64b08a8ac56f6ce'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:45.943920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-32e7f6df-fa9d-4822-9059-596241272922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ee5364270094596b64b08a8ac56f6ce'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:45.951018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-32e7f6df-fa9d-4822-9059-596241272922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ee5364270094596b64b08a8ac56f6ce'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:45.958239 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-32e7f6df-fa9d-4822-9059-596241272922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ee5364270094596b64b08a8ac56f6ce'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:45.969702 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-32e7f6df-fa9d-4822-9059-596241272922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ee5364270094596b64b08a8ac56f6ce'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:45.978046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1160/2312] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:03:45 2022] DELETE /identity/v3/users/8ee5364270094596b64b08a8ac56f6ce => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:46.171752 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ef19f-e544-45ae-b4da-20c78693fec3 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:46.184475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ef19f-e544-45ae-b4da-20c78693fec3 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:46.251453 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041bafc-3963-43a7-bedb-d6cd120e4514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:46.252038 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041bafc-3963-43a7-bedb-d6cd120e4514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:46.252450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041bafc-3963-43a7-bedb-d6cd120e4514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:46.253239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041bafc-3963-43a7-bedb-d6cd120e4514 admin admin] PATH_INFO: `/v3/projects/b5cb111df3ec4249a836518a598063a4` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:46.254379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041bafc-3963-43a7-bedb-d6cd120e4514 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5cb111df3ec4249a836518a598063a4)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:46.256337 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041bafc-3963-43a7-bedb-d6cd120e4514 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:46.277943 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:46.302769 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9041bafc-3963-43a7-bedb-d6cd120e4514 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5cb111df3ec4249a836518a598063a4', 'request_id': 'req-9041bafc-3963-43a7-bedb-d6cd120e4514'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:46.311043 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9041bafc-3963-43a7-bedb-d6cd120e4514 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5cb111df3ec4249a836518a598063a4', 'request_id': 'req-9041bafc-3963-43a7-bedb-d6cd120e4514'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:46.317418 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9041bafc-3963-43a7-bedb-d6cd120e4514 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5cb111df3ec4249a836518a598063a4', 'request_id': 'req-9041bafc-3963-43a7-bedb-d6cd120e4514'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:46.328618 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9041bafc-3963-43a7-bedb-d6cd120e4514 admin admin] The token cache is being invalidate because project b5cb111df3ec4249a836518a598063a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:03:46.328906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9041bafc-3963-43a7-bedb-d6cd120e4514 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:46.329547 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-9041bafc-3963-43a7-bedb-d6cd120e4514 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:46.331302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1153/2313] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:03:46 2022] DELETE /identity/v3/projects/b5cb111df3ec4249a836518a598063a4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:46.336359 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f882a1-9fa2-4c72-8fd0-c556fc4b3d98 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:46.336646 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f882a1-9fa2-4c72-8fd0-c556fc4b3d98 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:46.336942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f882a1-9fa2-4c72-8fd0-c556fc4b3d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:46.395963 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-59f882a1-9fa2-4c72-8fd0-c556fc4b3d98 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:46.396448 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-59f882a1-9fa2-4c72-8fd0-c556fc4b3d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:46.455588 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1161/2314] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:03:46.463742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26085b6-9352-423c-9ff4-ad693e547801 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:46.464036 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26085b6-9352-423c-9ff4-ad693e547801 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:46.464340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26085b6-9352-423c-9ff4-ad693e547801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:46.484347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b26085b6-9352-423c-9ff4-ad693e547801 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:46.485553 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b26085b6-9352-423c-9ff4-ad693e547801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:46.494338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1154/2315] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:46.498606 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f882a1-9fa2-4c72-8fd0-c556fc4b3d98 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:46.509443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f882a1-9fa2-4c72-8fd0-c556fc4b3d98 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:46.514357 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90beb53a-067d-4b4a-9a8b-08af76e930ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:46.514994 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90beb53a-067d-4b4a-9a8b-08af76e930ac admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:46.515277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90beb53a-067d-4b4a-9a8b-08af76e930ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:46.515550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90beb53a-067d-4b4a-9a8b-08af76e930ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:46.516233 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90beb53a-067d-4b4a-9a8b-08af76e930ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:03:46.516538 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90beb53a-067d-4b4a-9a8b-08af76e930ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:46.518683 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90beb53a-067d-4b4a-9a8b-08af76e930ac admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:46.530707 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1162/2316] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:03:46.535462 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26085b6-9352-423c-9ff4-ad693e547801 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:46.544677 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26085b6-9352-423c-9ff4-ad693e547801 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:46.549277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb4520-ea52-4ffb-a368-575fbc1869f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:46.550538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb4520-ea52-4ffb-a368-575fbc1869f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:46.550836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb4520-ea52-4ffb-a368-575fbc1869f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:46.551104 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb4520-ea52-4ffb-a368-575fbc1869f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:46.551721 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb4520-ea52-4ffb-a368-575fbc1869f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:46.553658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb4520-ea52-4ffb-a368-575fbc1869f5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:46.581143 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1155/2317] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:03:46 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:03:46.585373 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90beb53a-067d-4b4a-9a8b-08af76e930ac admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:46.590575 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:46.597362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90beb53a-067d-4b4a-9a8b-08af76e930ac admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:46.654149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05848b74-5bd6-40c7-8942-5b9829434595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:46.654782 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05848b74-5bd6-40c7-8942-5b9829434595 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:46.655059 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05848b74-5bd6-40c7-8942-5b9829434595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:46.655331 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05848b74-5bd6-40c7-8942-5b9829434595 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:46.655967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05848b74-5bd6-40c7-8942-5b9829434595 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:46.657918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05848b74-5bd6-40c7-8942-5b9829434595 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:46.680105 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1163/2318] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:03:46 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:03:46.684276 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb4520-ea52-4ffb-a368-575fbc1869f5 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:46.694993 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb4520-ea52-4ffb-a368-575fbc1869f5 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:46.699656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045b5bd-b5a8-4159-8566-f363e313b09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:46.700292 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045b5bd-b5a8-4159-8566-f363e313b09b admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:46.700571 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045b5bd-b5a8-4159-8566-f363e313b09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:46.700847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045b5bd-b5a8-4159-8566-f363e313b09b admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:46.701485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045b5bd-b5a8-4159-8566-f363e313b09b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:46.703077 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045b5bd-b5a8-4159-8566-f363e313b09b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:46.713456 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1156/2319] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:03:46.717681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05848b74-5bd6-40c7-8942-5b9829434595 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:46.728419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05848b74-5bd6-40c7-8942-5b9829434595 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:46.733732 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672e980-f396-4fd0-973f-7737920c326a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:46.734334 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672e980-f396-4fd0-973f-7737920c326a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:46.734630 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672e980-f396-4fd0-973f-7737920c326a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:46.734900 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672e980-f396-4fd0-973f-7737920c326a admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:46.735620 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672e980-f396-4fd0-973f-7737920c326a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:46.737894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672e980-f396-4fd0-973f-7737920c326a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:46.749826 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1164/2320] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:03:46.754573 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045b5bd-b5a8-4159-8566-f363e313b09b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:46.767690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045b5bd-b5a8-4159-8566-f363e313b09b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:46.772468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04485c8a-fa56-4e35-b38f-d620339f5f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:46.773130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04485c8a-fa56-4e35-b38f-d620339f5f5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:46.773417 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04485c8a-fa56-4e35-b38f-d620339f5f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:46.773694 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04485c8a-fa56-4e35-b38f-d620339f5f5a admin admin] PATH_INFO: `/v3/projects/6076fc86209b4369bb566aa468d70c9e/users/f2a6d4826a8f484497c1707af3d664fc/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:46.789152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04485c8a-fa56-4e35-b38f-d620339f5f5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6076fc86209b4369bb566aa468d70c9e, user_id=f2a6d4826a8f484497c1707af3d664fc, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:46.791775 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04485c8a-fa56-4e35-b38f-d620339f5f5a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:46.805068 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1157/2321] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:03:46 2022] PUT /identity/v3/projects/6076fc86209b4369bb566aa468d70c9e/users/f2a6d4826a8f484497c1707af3d664fc/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:46.816311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103169f5-685d-41b4-90e3-ccc403493da4 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:46.816508 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103169f5-685d-41b4-90e3-ccc403493da4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:46.816807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103169f5-685d-41b4-90e3-ccc403493da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:46.844087 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-103169f5-685d-41b4-90e3-ccc403493da4 None None] MFA Rules not processed for user `f2a6d4826a8f484497c1707af3d664fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:46.845310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-103169f5-685d-41b4-90e3-ccc403493da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:46.909437 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1165/2322] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:46.915036 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04485c8a-fa56-4e35-b38f-d620339f5f5a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:46.976965 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04485c8a-fa56-4e35-b38f-d620339f5f5a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:46.981596 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39049b-611f-464c-a4e3-089168804c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:46.982224 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39049b-611f-464c-a4e3-089168804c62 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:46.982501 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39049b-611f-464c-a4e3-089168804c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:46.982770 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39049b-611f-464c-a4e3-089168804c62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:46.983437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39049b-611f-464c-a4e3-089168804c62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:46.985397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39049b-611f-464c-a4e3-089168804c62 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:47.015943 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1158/2323] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:03:46 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:03:47.020039 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103169f5-685d-41b4-90e3-ccc403493da4 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:47.031110 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103169f5-685d-41b4-90e3-ccc403493da4 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:47.089408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671b731-af64-4086-8593-ee5f1c5e312d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:47.089940 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671b731-af64-4086-8593-ee5f1c5e312d admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:47.090216 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671b731-af64-4086-8593-ee5f1c5e312d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:47.090482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671b731-af64-4086-8593-ee5f1c5e312d admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:47.091109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4671b731-af64-4086-8593-ee5f1c5e312d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:47.093051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4671b731-af64-4086-8593-ee5f1c5e312d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:47.111526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:47.119130 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1166/2324] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:03:47 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:03:47.123009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39049b-611f-464c-a4e3-089168804c62 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:47.134988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39049b-611f-464c-a4e3-089168804c62 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:47.139978 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4460eb7-97a3-4f4c-94bd-899404c6a7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:47.140639 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4460eb7-97a3-4f4c-94bd-899404c6a7f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:47.140920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4460eb7-97a3-4f4c-94bd-899404c6a7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:47.141373 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4460eb7-97a3-4f4c-94bd-899404c6a7f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:47.142196 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4460eb7-97a3-4f4c-94bd-899404c6a7f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:47.144451 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4460eb7-97a3-4f4c-94bd-899404c6a7f8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:47.157303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1159/2325] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:03:47.162683 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671b731-af64-4086-8593-ee5f1c5e312d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:47.173096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671b731-af64-4086-8593-ee5f1c5e312d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:47.179263 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e95a71-51ad-4a1b-9d25-4dbc0dcf0a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:47.179937 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e95a71-51ad-4a1b-9d25-4dbc0dcf0a64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:47.180253 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e95a71-51ad-4a1b-9d25-4dbc0dcf0a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:47.180535 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e95a71-51ad-4a1b-9d25-4dbc0dcf0a64 admin admin] PATH_INFO: `/v3/projects/8f8fcbce5ae940129b59dc38b520e9b6/users/1700e4ca56714ced920ea0fdc0944203/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:47.199007 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e95a71-51ad-4a1b-9d25-4dbc0dcf0a64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f8fcbce5ae940129b59dc38b520e9b6, user_id=1700e4ca56714ced920ea0fdc0944203, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:47.201786 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e95a71-51ad-4a1b-9d25-4dbc0dcf0a64 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:47.214946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1167/2326] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:03:47 2022] PUT /identity/v3/projects/8f8fcbce5ae940129b59dc38b520e9b6/users/1700e4ca56714ced920ea0fdc0944203/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:03:47.218966 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4460eb7-97a3-4f4c-94bd-899404c6a7f8 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:47.229814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4460eb7-97a3-4f4c-94bd-899404c6a7f8 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:47.242674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:47.288618 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94a95e-927b-4547-b65d-f5324feee245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:47.289230 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94a95e-927b-4547-b65d-f5324feee245 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:47.289530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94a95e-927b-4547-b65d-f5324feee245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:47.289778 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94a95e-927b-4547-b65d-f5324feee245 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:47.290420 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94a95e-927b-4547-b65d-f5324feee245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:47.292009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94a95e-927b-4547-b65d-f5324feee245 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:47.302478 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1160/2327] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:03:47.307240 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e95a71-51ad-4a1b-9d25-4dbc0dcf0a64 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:47.317802 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e95a71-51ad-4a1b-9d25-4dbc0dcf0a64 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:47.322457 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66440261-258a-4992-b2fc-18e942c2b878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:47.323110 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66440261-258a-4992-b2fc-18e942c2b878 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:47.323391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66440261-258a-4992-b2fc-18e942c2b878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:47.323666 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66440261-258a-4992-b2fc-18e942c2b878 admin admin] PATH_INFO: `/v3/projects/8f8fcbce5ae940129b59dc38b520e9b6/users/1700e4ca56714ced920ea0fdc0944203/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:47.325747 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66440261-258a-4992-b2fc-18e942c2b878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f8fcbce5ae940129b59dc38b520e9b6, user_id=1700e4ca56714ced920ea0fdc0944203, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:47.328438 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66440261-258a-4992-b2fc-18e942c2b878 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:47.338904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1168/2328] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:03:47 2022] PUT /identity/v3/projects/8f8fcbce5ae940129b59dc38b520e9b6/users/1700e4ca56714ced920ea0fdc0944203/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:03:47.349976 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41584866-5838-4b53-9bfb-f1c212d09541 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:47.350192 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41584866-5838-4b53-9bfb-f1c212d09541 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:47.350506 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41584866-5838-4b53-9bfb-f1c212d09541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:47.381304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-41584866-5838-4b53-9bfb-f1c212d09541 None None] MFA Rules not processed for user `1700e4ca56714ced920ea0fdc0944203`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:47.382509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-41584866-5838-4b53-9bfb-f1c212d09541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:47.456618 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1161/2329] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:47 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:47.468154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66440261-258a-4992-b2fc-18e942c2b878 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:47.469886 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-66440261-258a-4992-b2fc-18e942c2b878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:47.529961 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66440261-258a-4992-b2fc-18e942c2b878 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:47.534337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f404a8-8747-43d9-85ce-851eaf452eae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:47.534993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f404a8-8747-43d9-85ce-851eaf452eae service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:47.535316 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f404a8-8747-43d9-85ce-851eaf452eae service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:47.535637 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f404a8-8747-43d9-85ce-851eaf452eae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:47.544697 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f404a8-8747-43d9-85ce-851eaf452eae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:47.546123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f404a8-8747-43d9-85ce-851eaf452eae service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:47.556160 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1169/2330] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:47.602257 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41584866-5838-4b53-9bfb-f1c212d09541 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:47.603584 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-41584866-5838-4b53-9bfb-f1c212d09541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:47.616512 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41584866-5838-4b53-9bfb-f1c212d09541 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:47.620049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1a8df-091c-49ad-a507-c983c9aa2774 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:47.620666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1a8df-091c-49ad-a507-c983c9aa2774 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:47.620944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1a8df-091c-49ad-a507-c983c9aa2774 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:47.621207 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1a8df-091c-49ad-a507-c983c9aa2774 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:47.677312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee1a8df-091c-49ad-a507-c983c9aa2774 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:47.678762 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee1a8df-091c-49ad-a507-c983c9aa2774 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:47.687776 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1162/2331] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:51.082799 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f404a8-8747-43d9-85ce-851eaf452eae service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:51.145140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f404a8-8747-43d9-85ce-851eaf452eae service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:51.149806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ffbdf-2842-45a3-8e33-4704cb06b260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:51.150432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ffbdf-2842-45a3-8e33-4704cb06b260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:51.150967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ffbdf-2842-45a3-8e33-4704cb06b260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:51.151243 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ffbdf-2842-45a3-8e33-4704cb06b260 admin admin] PATH_INFO: `/v3/users/f2a6d4826a8f484497c1707af3d664fc` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:51.152411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ffbdf-2842-45a3-8e33-4704cb06b260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2a6d4826a8f484497c1707af3d664fc)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:51.154324 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ffbdf-2842-45a3-8e33-4704cb06b260 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:51.202078 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-dc3ffbdf-2842-45a3-8e33-4704cb06b260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2a6d4826a8f484497c1707af3d664fc'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.202743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-dc3ffbdf-2842-45a3-8e33-4704cb06b260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2a6d4826a8f484497c1707af3d664fc'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.211719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-dc3ffbdf-2842-45a3-8e33-4704cb06b260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2a6d4826a8f484497c1707af3d664fc'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.219206 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-dc3ffbdf-2842-45a3-8e33-4704cb06b260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2a6d4826a8f484497c1707af3d664fc'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.225949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:51.235261 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-dc3ffbdf-2842-45a3-8e33-4704cb06b260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2a6d4826a8f484497c1707af3d664fc'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.242242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1170/2332] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:03:51 2022] DELETE /identity/v3/users/f2a6d4826a8f484497c1707af3d664fc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:51.246854 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d55bb-87ac-48fc-9c2a-69bb4148641c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:51.247338 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d55bb-87ac-48fc-9c2a-69bb4148641c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:51.247507 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d55bb-87ac-48fc-9c2a-69bb4148641c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:51.266356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9f1d55bb-87ac-48fc-9c2a-69bb4148641c None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:51.267608 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9f1d55bb-87ac-48fc-9c2a-69bb4148641c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:51.302685 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:51.329669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1163/2333] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:51.340233 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ffbdf-2842-45a3-8e33-4704cb06b260 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:51.341643 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-dc3ffbdf-2842-45a3-8e33-4704cb06b260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:51.420801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ffbdf-2842-45a3-8e33-4704cb06b260 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:51.424446 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37eaf80-029f-430c-be15-603e79619dc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:51.425175 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37eaf80-029f-430c-be15-603e79619dc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:51.425456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37eaf80-029f-430c-be15-603e79619dc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:51.425730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37eaf80-029f-430c-be15-603e79619dc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:51.434730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37eaf80-029f-430c-be15-603e79619dc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:51.436165 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37eaf80-029f-430c-be15-603e79619dc1 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:51.445773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1171/2334] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:51.724732 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d55bb-87ac-48fc-9c2a-69bb4148641c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:51.726214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9f1d55bb-87ac-48fc-9c2a-69bb4148641c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:51.739472 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d55bb-87ac-48fc-9c2a-69bb4148641c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:51.744742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de849c90-68b7-429f-82f3-51d75ca43604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:51.744869 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de849c90-68b7-429f-82f3-51d75ca43604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:51.745125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de849c90-68b7-429f-82f3-51d75ca43604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:51.745400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de849c90-68b7-429f-82f3-51d75ca43604 admin admin] PATH_INFO: `/v3/projects/6076fc86209b4369bb566aa468d70c9e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:51.746545 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de849c90-68b7-429f-82f3-51d75ca43604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6076fc86209b4369bb566aa468d70c9e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:51.748573 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de849c90-68b7-429f-82f3-51d75ca43604 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:51.802179 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-de849c90-68b7-429f-82f3-51d75ca43604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6076fc86209b4369bb566aa468d70c9e', 'request_id': 'req-de849c90-68b7-429f-82f3-51d75ca43604'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.809418 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-de849c90-68b7-429f-82f3-51d75ca43604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6076fc86209b4369bb566aa468d70c9e', 'request_id': 'req-de849c90-68b7-429f-82f3-51d75ca43604'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.815894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-de849c90-68b7-429f-82f3-51d75ca43604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6076fc86209b4369bb566aa468d70c9e', 'request_id': 'req-de849c90-68b7-429f-82f3-51d75ca43604'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.826743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-de849c90-68b7-429f-82f3-51d75ca43604 admin admin] The token cache is being invalidate because project 6076fc86209b4369bb566aa468d70c9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:03:51.827027 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-de849c90-68b7-429f-82f3-51d75ca43604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.827672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-de849c90-68b7-429f-82f3-51d75ca43604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.829207 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1164/2335] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:03:51 2022] DELETE /identity/v3/projects/6076fc86209b4369bb566aa468d70c9e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:51.834106 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37eaf80-029f-430c-be15-603e79619dc1 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:51.843641 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37eaf80-029f-430c-be15-603e79619dc1 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:51.899988 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404e846-bd28-4466-af04-b0b3f8a2cf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:51.900653 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404e846-bd28-4466-af04-b0b3f8a2cf4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:51.901189 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404e846-bd28-4466-af04-b0b3f8a2cf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:51.901462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404e846-bd28-4466-af04-b0b3f8a2cf4d admin admin] PATH_INFO: `/v3/users/1700e4ca56714ced920ea0fdc0944203` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:51.902588 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404e846-bd28-4466-af04-b0b3f8a2cf4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1700e4ca56714ced920ea0fdc0944203)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:51.904542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404e846-bd28-4466-af04-b0b3f8a2cf4d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:51.951728 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0404e846-bd28-4466-af04-b0b3f8a2cf4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1700e4ca56714ced920ea0fdc0944203'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.952394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0404e846-bd28-4466-af04-b0b3f8a2cf4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1700e4ca56714ced920ea0fdc0944203'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.961722 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0404e846-bd28-4466-af04-b0b3f8a2cf4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1700e4ca56714ced920ea0fdc0944203'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.968766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0404e846-bd28-4466-af04-b0b3f8a2cf4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1700e4ca56714ced920ea0fdc0944203'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.973973 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:51.982185 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0404e846-bd28-4466-af04-b0b3f8a2cf4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1700e4ca56714ced920ea0fdc0944203'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:51.988809 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1172/2336] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:03:51 2022] DELETE /identity/v3/users/1700e4ca56714ced920ea0fdc0944203 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:52.263837 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de849c90-68b7-429f-82f3-51d75ca43604 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:52.266021 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-de849c90-68b7-429f-82f3-51d75ca43604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:52.282905 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de849c90-68b7-429f-82f3-51d75ca43604 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:52.341614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d1890-9937-4627-9e10-70a1c487ecd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:52.342060 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d1890-9937-4627-9e10-70a1c487ecd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:52.342340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d1890-9937-4627-9e10-70a1c487ecd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:52.342712 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d1890-9937-4627-9e10-70a1c487ecd6 admin admin] PATH_INFO: `/v3/projects/8f8fcbce5ae940129b59dc38b520e9b6` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:52.343731 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d1890-9937-4627-9e10-70a1c487ecd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f8fcbce5ae940129b59dc38b520e9b6)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:52.345678 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d1890-9937-4627-9e10-70a1c487ecd6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:52.391261 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e75d1890-9937-4627-9e10-70a1c487ecd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f8fcbce5ae940129b59dc38b520e9b6', 'request_id': 'req-e75d1890-9937-4627-9e10-70a1c487ecd6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:52.398525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e75d1890-9937-4627-9e10-70a1c487ecd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f8fcbce5ae940129b59dc38b520e9b6', 'request_id': 'req-e75d1890-9937-4627-9e10-70a1c487ecd6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:52.403356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:52.404688 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e75d1890-9937-4627-9e10-70a1c487ecd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f8fcbce5ae940129b59dc38b520e9b6', 'request_id': 'req-e75d1890-9937-4627-9e10-70a1c487ecd6'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:52.417284 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e75d1890-9937-4627-9e10-70a1c487ecd6 admin admin] The token cache is being invalidate because project 8f8fcbce5ae940129b59dc38b520e9b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:03:52.417570 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e75d1890-9937-4627-9e10-70a1c487ecd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:52.418215 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e75d1890-9937-4627-9e10-70a1c487ecd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:52.419737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1165/2337] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:03:52 2022] DELETE /identity/v3/projects/8f8fcbce5ae940129b59dc38b520e9b6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:52.424518 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd811d07-4007-4444-84fb-998ebf2bc248 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:52.425675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd811d07-4007-4444-84fb-998ebf2bc248 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:52.425675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd811d07-4007-4444-84fb-998ebf2bc248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:52.445493 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-bd811d07-4007-4444-84fb-998ebf2bc248 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:52.446703 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bd811d07-4007-4444-84fb-998ebf2bc248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:52.507800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1173/2338] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:52.515799 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63005c02-718f-406c-af87-eead121311e4 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:52.516201 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63005c02-718f-406c-af87-eead121311e4 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:52.516477 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63005c02-718f-406c-af87-eead121311e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:52.536695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-63005c02-718f-406c-af87-eead121311e4 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:52.537877 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-63005c02-718f-406c-af87-eead121311e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:52.545742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1166/2339] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:52.550594 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd811d07-4007-4444-84fb-998ebf2bc248 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:52.560231 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd811d07-4007-4444-84fb-998ebf2bc248 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:52.564831 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f27fd-e33f-4f70-a40c-c80734bbae72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:52.565463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f27fd-e33f-4f70-a40c-c80734bbae72 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:52.565737 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f27fd-e33f-4f70-a40c-c80734bbae72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:52.566013 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f27fd-e33f-4f70-a40c-c80734bbae72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:52.566650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f27fd-e33f-4f70-a40c-c80734bbae72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:03:52.566973 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f27fd-e33f-4f70-a40c-c80734bbae72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:52.568596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f27fd-e33f-4f70-a40c-c80734bbae72 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:52.580161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1174/2340] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:03:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:03:52.584643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63005c02-718f-406c-af87-eead121311e4 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:52.593807 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63005c02-718f-406c-af87-eead121311e4 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:52.599039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a06251-413e-466f-bff0-e5bf2570ad8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:52.599643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a06251-413e-466f-bff0-e5bf2570ad8f admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:52.599920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a06251-413e-466f-bff0-e5bf2570ad8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:52.600224 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a06251-413e-466f-bff0-e5bf2570ad8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:52.600855 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a06251-413e-466f-bff0-e5bf2570ad8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:52.602739 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a06251-413e-466f-bff0-e5bf2570ad8f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:52.630315 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1167/2341] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:03:52 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:03:52.635121 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f27fd-e33f-4f70-a40c-c80734bbae72 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:52.645360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f27fd-e33f-4f70-a40c-c80734bbae72 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:52.702867 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d01e6-0991-4a25-81ca-542eea807307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:52.703473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d01e6-0991-4a25-81ca-542eea807307 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:52.703752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d01e6-0991-4a25-81ca-542eea807307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:52.704017 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d01e6-0991-4a25-81ca-542eea807307 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:52.704663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d01e6-0991-4a25-81ca-542eea807307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:52.706613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d01e6-0991-4a25-81ca-542eea807307 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:52.728797 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1175/2342] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:03:52 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:03:52.733167 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a06251-413e-466f-bff0-e5bf2570ad8f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:52.744583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a06251-413e-466f-bff0-e5bf2570ad8f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:52.749308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b882d3-7861-4df0-be55-70e66cf866a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:52.749909 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b882d3-7861-4df0-be55-70e66cf866a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:52.750191 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b882d3-7861-4df0-be55-70e66cf866a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:52.750464 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b882d3-7861-4df0-be55-70e66cf866a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:52.751127 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b882d3-7861-4df0-be55-70e66cf866a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:52.752762 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b882d3-7861-4df0-be55-70e66cf866a4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:52.762725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1168/2343] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:03:52.766785 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d01e6-0991-4a25-81ca-542eea807307 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:52.777288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d01e6-0991-4a25-81ca-542eea807307 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:52.782166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be3edb-053b-470c-a34d-826a3e8a5e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:52.782719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be3edb-053b-470c-a34d-826a3e8a5e66 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:52.783000 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be3edb-053b-470c-a34d-826a3e8a5e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:52.783347 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be3edb-053b-470c-a34d-826a3e8a5e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:52.784558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be3edb-053b-470c-a34d-826a3e8a5e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:52.786385 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be3edb-053b-470c-a34d-826a3e8a5e66 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:52.797247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1176/2344] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:03:52.802107 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b882d3-7861-4df0-be55-70e66cf866a4 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:52.812366 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b882d3-7861-4df0-be55-70e66cf866a4 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:52.817213 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b8f44-060c-4fa7-bec8-1362279bbe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:52.817862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b8f44-060c-4fa7-bec8-1362279bbe60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:52.818141 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b8f44-060c-4fa7-bec8-1362279bbe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:52.818409 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b8f44-060c-4fa7-bec8-1362279bbe60 admin admin] PATH_INFO: `/v3/projects/441da82fa0d9423b964ef31c678b5980/users/f5ae8279fd4c44e2a00b5f92aa7c454a/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:52.833410 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b8f44-060c-4fa7-bec8-1362279bbe60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=441da82fa0d9423b964ef31c678b5980, user_id=f5ae8279fd4c44e2a00b5f92aa7c454a, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:52.836070 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b8f44-060c-4fa7-bec8-1362279bbe60 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:52.849639 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1169/2345] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:03:52 2022] PUT /identity/v3/projects/441da82fa0d9423b964ef31c678b5980/users/f5ae8279fd4c44e2a00b5f92aa7c454a/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:52.861011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a512e8b-ef57-4266-b678-d08886eb9e34 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:52.861297 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a512e8b-ef57-4266-b678-d08886eb9e34 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:52.861570 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a512e8b-ef57-4266-b678-d08886eb9e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:52.888310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6a512e8b-ef57-4266-b678-d08886eb9e34 None None] MFA Rules not processed for user `f5ae8279fd4c44e2a00b5f92aa7c454a`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:52.889518 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6a512e8b-ef57-4266-b678-d08886eb9e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:52.930958 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:52.954013 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1177/2346] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:52.959515 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b8f44-060c-4fa7-bec8-1362279bbe60 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:53.022637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b8f44-060c-4fa7-bec8-1362279bbe60 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:53.027656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3327a-093e-4396-ba55-9a528d12c7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:53.028308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3327a-093e-4396-ba55-9a528d12c7de admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:53.028594 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3327a-093e-4396-ba55-9a528d12c7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:53.028873 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3327a-093e-4396-ba55-9a528d12c7de admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:53.029517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3327a-093e-4396-ba55-9a528d12c7de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:53.031413 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3327a-093e-4396-ba55-9a528d12c7de admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:53.058214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1170/2347] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:03:52 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:03:53.062311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a512e8b-ef57-4266-b678-d08886eb9e34 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:53.073040 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a512e8b-ef57-4266-b678-d08886eb9e34 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:53.131286 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c841b-7d03-4105-882b-6b3a7b16280d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:53.131806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c841b-7d03-4105-882b-6b3a7b16280d admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:53.132150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c841b-7d03-4105-882b-6b3a7b16280d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:53.132430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c841b-7d03-4105-882b-6b3a7b16280d admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:53.133146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c841b-7d03-4105-882b-6b3a7b16280d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:53.135106 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c841b-7d03-4105-882b-6b3a7b16280d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:53.156782 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1178/2348] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:03:53 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:03:53.161381 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3327a-093e-4396-ba55-9a528d12c7de admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:53.171813 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3327a-093e-4396-ba55-9a528d12c7de admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:53.176617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c467d-4049-440f-af63-627cf306ea06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:53.177222 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c467d-4049-440f-af63-627cf306ea06 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:53.177502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c467d-4049-440f-af63-627cf306ea06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:53.177772 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c467d-4049-440f-af63-627cf306ea06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:53.178442 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c467d-4049-440f-af63-627cf306ea06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:53.180067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c467d-4049-440f-af63-627cf306ea06 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:53.191488 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1171/2349] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:03:53.195823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c841b-7d03-4105-882b-6b3a7b16280d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:53.206444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c841b-7d03-4105-882b-6b3a7b16280d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:53.212612 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52e5ee-1335-4198-ae81-36b86a4118ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:53.213362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52e5ee-1335-4198-ae81-36b86a4118ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:53.213538 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52e5ee-1335-4198-ae81-36b86a4118ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:53.213811 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52e5ee-1335-4198-ae81-36b86a4118ef admin admin] PATH_INFO: `/v3/projects/77043bb0f56246ccab96fd90dcabfe36/users/1d793a29697440899df317e1d9453741/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:53.229445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52e5ee-1335-4198-ae81-36b86a4118ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=77043bb0f56246ccab96fd90dcabfe36, user_id=1d793a29697440899df317e1d9453741, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:53.232384 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52e5ee-1335-4198-ae81-36b86a4118ef admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:53.245719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1179/2350] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:03:53 2022] PUT /identity/v3/projects/77043bb0f56246ccab96fd90dcabfe36/users/1d793a29697440899df317e1d9453741/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:53.249657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c467d-4049-440f-af63-627cf306ea06 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:53.260285 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c467d-4049-440f-af63-627cf306ea06 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:53.317148 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6fd17e-e0ee-4265-ac39-e3baa1c51890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:53.317818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6fd17e-e0ee-4265-ac39-e3baa1c51890 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:53.318035 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6fd17e-e0ee-4265-ac39-e3baa1c51890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:53.318308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6fd17e-e0ee-4265-ac39-e3baa1c51890 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:53.318952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6fd17e-e0ee-4265-ac39-e3baa1c51890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:53.321105 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6fd17e-e0ee-4265-ac39-e3baa1c51890 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:53.323622 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:53.333704 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1172/2351] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:03:53.337670 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52e5ee-1335-4198-ae81-36b86a4118ef admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:53.347809 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52e5ee-1335-4198-ae81-36b86a4118ef admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:53.352422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8708dfc-ec74-4300-90a5-f456b17d8f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:53.353069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8708dfc-ec74-4300-90a5-f456b17d8f06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:53.353433 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8708dfc-ec74-4300-90a5-f456b17d8f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:53.353777 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8708dfc-ec74-4300-90a5-f456b17d8f06 admin admin] PATH_INFO: `/v3/projects/77043bb0f56246ccab96fd90dcabfe36/users/1d793a29697440899df317e1d9453741/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:53.356015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8708dfc-ec74-4300-90a5-f456b17d8f06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77043bb0f56246ccab96fd90dcabfe36, user_id=1d793a29697440899df317e1d9453741, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:53.358421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8708dfc-ec74-4300-90a5-f456b17d8f06 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:53.369328 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1180/2352] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:03:53 2022] PUT /identity/v3/projects/77043bb0f56246ccab96fd90dcabfe36/users/1d793a29697440899df317e1d9453741/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:53.379711 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4b0cc-8d60-4f6f-a4eb-510866b1a1a0 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:53.379999 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4b0cc-8d60-4f6f-a4eb-510866b1a1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:53.380301 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4b0cc-8d60-4f6f-a4eb-510866b1a1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:53.408931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9fb4b0cc-8d60-4f6f-a4eb-510866b1a1a0 None None] MFA Rules not processed for user `1d793a29697440899df317e1d9453741`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:53.410209 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9fb4b0cc-8d60-4f6f-a4eb-510866b1a1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:53.477537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1173/2353] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:53 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:53.488573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8708dfc-ec74-4300-90a5-f456b17d8f06 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:53.490386 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c8708dfc-ec74-4300-90a5-f456b17d8f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:53.548738 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8708dfc-ec74-4300-90a5-f456b17d8f06 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:53.553000 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7b2c8-a9d6-4efe-9e6c-200156e268d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:53.553637 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7b2c8-a9d6-4efe-9e6c-200156e268d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:53.553920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7b2c8-a9d6-4efe-9e6c-200156e268d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:53.554193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7b2c8-a9d6-4efe-9e6c-200156e268d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:53.563286 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7b2c8-a9d6-4efe-9e6c-200156e268d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:53.564744 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7b2c8-a9d6-4efe-9e6c-200156e268d8 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:53.574429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1181/2354] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:53 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:03:54.164313 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4b0cc-8d60-4f6f-a4eb-510866b1a1a0 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:54.229038 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4b0cc-8d60-4f6f-a4eb-510866b1a1a0 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:54.233568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764f1bb-686b-4cea-8a15-0d66763e6028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:54.234196 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764f1bb-686b-4cea-8a15-0d66763e6028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:54.234477 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764f1bb-686b-4cea-8a15-0d66763e6028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:54.234751 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764f1bb-686b-4cea-8a15-0d66763e6028 admin admin] PATH_INFO: `/v3/users/f5ae8279fd4c44e2a00b5f92aa7c454a` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:54.235867 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764f1bb-686b-4cea-8a15-0d66763e6028 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5ae8279fd4c44e2a00b5f92aa7c454a)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:54.238054 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764f1bb-686b-4cea-8a15-0d66763e6028 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:54.286813 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5764f1bb-686b-4cea-8a15-0d66763e6028 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5ae8279fd4c44e2a00b5f92aa7c454a'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:54.293457 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5764f1bb-686b-4cea-8a15-0d66763e6028 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5ae8279fd4c44e2a00b5f92aa7c454a'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:54.305719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5764f1bb-686b-4cea-8a15-0d66763e6028 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5ae8279fd4c44e2a00b5f92aa7c454a'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:54.312517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5764f1bb-686b-4cea-8a15-0d66763e6028 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5ae8279fd4c44e2a00b5f92aa7c454a'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:54.313200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5764f1bb-686b-4cea-8a15-0d66763e6028 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5ae8279fd4c44e2a00b5f92aa7c454a'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:54.320456 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1174/2355] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:03:54 2022] DELETE /identity/v3/users/f5ae8279fd4c44e2a00b5f92aa7c454a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:54.324928 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3af29-3049-4cc3-a0e6-4e4baae38d99 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:54.325214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3af29-3049-4cc3-a0e6-4e4baae38d99 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:54.325484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3af29-3049-4cc3-a0e6-4e4baae38d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:54.344591 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-32a3af29-3049-4cc3-a0e6-4e4baae38d99 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:54.345783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-32a3af29-3049-4cc3-a0e6-4e4baae38d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:54.378428 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:54.406372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1182/2356] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:54.416237 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764f1bb-686b-4cea-8a15-0d66763e6028 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:54.417962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5764f1bb-686b-4cea-8a15-0d66763e6028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:54.423505 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:54.479433 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764f1bb-686b-4cea-8a15-0d66763e6028 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:54.484347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aefdd0-b14c-4a04-af2b-cf433cf29d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:54.485013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aefdd0-b14c-4a04-af2b-cf433cf29d06 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:54.485292 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aefdd0-b14c-4a04-af2b-cf433cf29d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:54.485559 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aefdd0-b14c-4a04-af2b-cf433cf29d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:54.495470 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aefdd0-b14c-4a04-af2b-cf433cf29d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:54.496919 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aefdd0-b14c-4a04-af2b-cf433cf29d06 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:54.506966 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1175/2357] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:54.788051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3af29-3049-4cc3-a0e6-4e4baae38d99 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:54.789510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-32a3af29-3049-4cc3-a0e6-4e4baae38d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:54.802082 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3af29-3049-4cc3-a0e6-4e4baae38d99 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:54.806992 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee9df2-1871-4e9b-ab94-8b8fd4ae25fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:54.807632 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee9df2-1871-4e9b-ab94-8b8fd4ae25fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:54.807911 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee9df2-1871-4e9b-ab94-8b8fd4ae25fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:54.808220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee9df2-1871-4e9b-ab94-8b8fd4ae25fe admin admin] PATH_INFO: `/v3/projects/441da82fa0d9423b964ef31c678b5980` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:54.809344 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee9df2-1871-4e9b-ab94-8b8fd4ae25fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=441da82fa0d9423b964ef31c678b5980)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:54.811329 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee9df2-1871-4e9b-ab94-8b8fd4ae25fe admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:54.860840 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-40ee9df2-1871-4e9b-ab94-8b8fd4ae25fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '441da82fa0d9423b964ef31c678b5980', 'request_id': 'req-40ee9df2-1871-4e9b-ab94-8b8fd4ae25fe'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:54.866422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-40ee9df2-1871-4e9b-ab94-8b8fd4ae25fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '441da82fa0d9423b964ef31c678b5980', 'request_id': 'req-40ee9df2-1871-4e9b-ab94-8b8fd4ae25fe'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:54.874085 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-40ee9df2-1871-4e9b-ab94-8b8fd4ae25fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '441da82fa0d9423b964ef31c678b5980', 'request_id': 'req-40ee9df2-1871-4e9b-ab94-8b8fd4ae25fe'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:54.886713 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-40ee9df2-1871-4e9b-ab94-8b8fd4ae25fe admin admin] The token cache is being invalidate because project 441da82fa0d9423b964ef31c678b5980 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:03:54.887002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-40ee9df2-1871-4e9b-ab94-8b8fd4ae25fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:54.887673 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-40ee9df2-1871-4e9b-ab94-8b8fd4ae25fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:54.889233 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1183/2358] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:03:54 2022] DELETE /identity/v3/projects/441da82fa0d9423b964ef31c678b5980 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:03:54.893258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aefdd0-b14c-4a04-af2b-cf433cf29d06 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:54.903799 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aefdd0-b14c-4a04-af2b-cf433cf29d06 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:54.959327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d540c-9f10-4378-aaa5-19f731bfaec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:54.959960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d540c-9f10-4378-aaa5-19f731bfaec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:54.960288 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d540c-9f10-4378-aaa5-19f731bfaec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:54.960563 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d540c-9f10-4378-aaa5-19f731bfaec7 admin admin] PATH_INFO: `/v3/users/1d793a29697440899df317e1d9453741` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:54.961713 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d540c-9f10-4378-aaa5-19f731bfaec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d793a29697440899df317e1d9453741)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:54.963665 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d540c-9f10-4378-aaa5-19f731bfaec7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:55.013190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bb9d540c-9f10-4378-aaa5-19f731bfaec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d793a29697440899df317e1d9453741'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:55.021017 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bb9d540c-9f10-4378-aaa5-19f731bfaec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d793a29697440899df317e1d9453741'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:55.033130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bb9d540c-9f10-4378-aaa5-19f731bfaec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d793a29697440899df317e1d9453741'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:55.040514 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bb9d540c-9f10-4378-aaa5-19f731bfaec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d793a29697440899df317e1d9453741'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:55.041168 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-bb9d540c-9f10-4378-aaa5-19f731bfaec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d793a29697440899df317e1d9453741'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:55.046905 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1176/2359] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:03:54 2022] DELETE /identity/v3/users/1d793a29697440899df317e1d9453741 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:55.326961 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee9df2-1871-4e9b-ab94-8b8fd4ae25fe admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:55.328510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-40ee9df2-1871-4e9b-ab94-8b8fd4ae25fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:55.341323 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee9df2-1871-4e9b-ab94-8b8fd4ae25fe admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:55.401418 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4863ff-15e7-4158-99ba-62eff87b4cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:55.402056 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4863ff-15e7-4158-99ba-62eff87b4cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:55.402356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4863ff-15e7-4158-99ba-62eff87b4cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:55.402627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4863ff-15e7-4158-99ba-62eff87b4cc9 admin admin] PATH_INFO: `/v3/projects/77043bb0f56246ccab96fd90dcabfe36` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:55.403739 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4863ff-15e7-4158-99ba-62eff87b4cc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77043bb0f56246ccab96fd90dcabfe36)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:55.405644 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4863ff-15e7-4158-99ba-62eff87b4cc9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:55.453540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bc4863ff-15e7-4158-99ba-62eff87b4cc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77043bb0f56246ccab96fd90dcabfe36', 'request_id': 'req-bc4863ff-15e7-4158-99ba-62eff87b4cc9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:55.459455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bc4863ff-15e7-4158-99ba-62eff87b4cc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77043bb0f56246ccab96fd90dcabfe36', 'request_id': 'req-bc4863ff-15e7-4158-99ba-62eff87b4cc9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:55.463214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:55.467119 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bc4863ff-15e7-4158-99ba-62eff87b4cc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77043bb0f56246ccab96fd90dcabfe36', 'request_id': 'req-bc4863ff-15e7-4158-99ba-62eff87b4cc9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:55.483104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bc4863ff-15e7-4158-99ba-62eff87b4cc9 admin admin] The token cache is being invalidate because project 77043bb0f56246ccab96fd90dcabfe36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:03:55.483448 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bc4863ff-15e7-4158-99ba-62eff87b4cc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:55.484176 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bc4863ff-15e7-4158-99ba-62eff87b4cc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:55.485799 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1184/2360] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:03:55 2022] DELETE /identity/v3/projects/77043bb0f56246ccab96fd90dcabfe36 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:55.491958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee2b7e-0919-4903-acc1-da5a644ce781 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:55.492511 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee2b7e-0919-4903-acc1-da5a644ce781 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:55.492608 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee2b7e-0919-4903-acc1-da5a644ce781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:55.512400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-98ee2b7e-0919-4903-acc1-da5a644ce781 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:55.513607 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-98ee2b7e-0919-4903-acc1-da5a644ce781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:55.532192 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:55.576811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1177/2361] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:55.586138 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a195257-4936-4bbb-b6b6-bcc6c250bc7d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:55.586482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a195257-4936-4bbb-b6b6-bcc6c250bc7d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:55.586710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a195257-4936-4bbb-b6b6-bcc6c250bc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:55.607295 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9a195257-4936-4bbb-b6b6-bcc6c250bc7d None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:55.608537 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9a195257-4936-4bbb-b6b6-bcc6c250bc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:55.617460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1185/2362] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:55.621710 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee2b7e-0919-4903-acc1-da5a644ce781 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:55.631602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee2b7e-0919-4903-acc1-da5a644ce781 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:55.636207 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae2d6e-debd-40d6-b86f-fc4064946df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:55.636808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae2d6e-debd-40d6-b86f-fc4064946df7 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:55.637111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae2d6e-debd-40d6-b86f-fc4064946df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:55.637358 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae2d6e-debd-40d6-b86f-fc4064946df7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:55.638023 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae2d6e-debd-40d6-b86f-fc4064946df7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:03:55.638327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae2d6e-debd-40d6-b86f-fc4064946df7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:55.639914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae2d6e-debd-40d6-b86f-fc4064946df7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:55.651599 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1178/2363] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:03:55.655824 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a195257-4936-4bbb-b6b6-bcc6c250bc7d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:55.665837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a195257-4936-4bbb-b6b6-bcc6c250bc7d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:55.670542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1809a21-c96f-4d7d-9c3a-28f2b7604b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:55.671152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1809a21-c96f-4d7d-9c3a-28f2b7604b5b admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:55.671415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1809a21-c96f-4d7d-9c3a-28f2b7604b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:55.671688 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1809a21-c96f-4d7d-9c3a-28f2b7604b5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:55.672347 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1809a21-c96f-4d7d-9c3a-28f2b7604b5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:55.674240 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1809a21-c96f-4d7d-9c3a-28f2b7604b5b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:55.701239 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1186/2364] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:03:55 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:03:55.705695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae2d6e-debd-40d6-b86f-fc4064946df7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:55.715709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae2d6e-debd-40d6-b86f-fc4064946df7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:55.773450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9932f-d019-4ecd-9420-be4d4322c82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:55.774059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9932f-d019-4ecd-9420-be4d4322c82f admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:55.774343 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9932f-d019-4ecd-9420-be4d4322c82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:55.774618 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9932f-d019-4ecd-9420-be4d4322c82f admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:55.775235 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9932f-d019-4ecd-9420-be4d4322c82f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:55.777460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9932f-d019-4ecd-9420-be4d4322c82f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:55.800827 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1179/2365] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:03:55 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:03:55.805193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1809a21-c96f-4d7d-9c3a-28f2b7604b5b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:55.815413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1809a21-c96f-4d7d-9c3a-28f2b7604b5b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:55.820519 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae389a6a-1063-424e-8f26-4e73f4e04241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:55.821146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae389a6a-1063-424e-8f26-4e73f4e04241 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:55.821403 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae389a6a-1063-424e-8f26-4e73f4e04241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:55.821675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae389a6a-1063-424e-8f26-4e73f4e04241 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:55.822354 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae389a6a-1063-424e-8f26-4e73f4e04241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:55.823950 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae389a6a-1063-424e-8f26-4e73f4e04241 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:55.835494 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1187/2366] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:03:55.839606 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9932f-d019-4ecd-9420-be4d4322c82f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:55.850523 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9932f-d019-4ecd-9420-be4d4322c82f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:55.855341 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9010b0-48b6-4bac-aef2-926260ae0c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:55.855940 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9010b0-48b6-4bac-aef2-926260ae0c45 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:55.856259 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9010b0-48b6-4bac-aef2-926260ae0c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:55.856533 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9010b0-48b6-4bac-aef2-926260ae0c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:55.857176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9010b0-48b6-4bac-aef2-926260ae0c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:55.858753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9010b0-48b6-4bac-aef2-926260ae0c45 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:55.869540 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1180/2367] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:03:55.873668 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae389a6a-1063-424e-8f26-4e73f4e04241 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:55.883608 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae389a6a-1063-424e-8f26-4e73f4e04241 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:55.888374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a99012-bb4b-4bea-a6e9-a260345f0a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:55.889024 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a99012-bb4b-4bea-a6e9-a260345f0a5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:55.889302 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a99012-bb4b-4bea-a6e9-a260345f0a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:55.889575 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a99012-bb4b-4bea-a6e9-a260345f0a5f admin admin] PATH_INFO: `/v3/projects/e100437bdebd4b78ab0ddddc2c61ef8d/users/38b18b8deff0429a9743d56f72c7cd4e/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:55.905265 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a99012-bb4b-4bea-a6e9-a260345f0a5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e100437bdebd4b78ab0ddddc2c61ef8d, user_id=38b18b8deff0429a9743d56f72c7cd4e, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:55.907969 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a99012-bb4b-4bea-a6e9-a260345f0a5f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:55.921463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1188/2368] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:03:55 2022] PUT /identity/v3/projects/e100437bdebd4b78ab0ddddc2c61ef8d/users/38b18b8deff0429a9743d56f72c7cd4e/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:55.931782 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081731ac-3cfa-4e4f-9d3d-4540fc5239fe None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:55.932137 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081731ac-3cfa-4e4f-9d3d-4540fc5239fe None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:55.932415 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081731ac-3cfa-4e4f-9d3d-4540fc5239fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:55.959688 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-081731ac-3cfa-4e4f-9d3d-4540fc5239fe None None] MFA Rules not processed for user `38b18b8deff0429a9743d56f72c7cd4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:55.960948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-081731ac-3cfa-4e4f-9d3d-4540fc5239fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:56.029638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1181/2369] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:56.034173 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a99012-bb4b-4bea-a6e9-a260345f0a5f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:56.098485 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a99012-bb4b-4bea-a6e9-a260345f0a5f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:56.103264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b819e10-3791-47b7-af84-8e91389992d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:56.103872 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b819e10-3791-47b7-af84-8e91389992d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:56.104198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b819e10-3791-47b7-af84-8e91389992d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:56.104469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b819e10-3791-47b7-af84-8e91389992d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:56.105095 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b819e10-3791-47b7-af84-8e91389992d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:56.107021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b819e10-3791-47b7-af84-8e91389992d9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:56.137032 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1189/2370] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:03:56 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:03:56.140404 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081731ac-3cfa-4e4f-9d3d-4540fc5239fe None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:56.151918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081731ac-3cfa-4e4f-9d3d-4540fc5239fe None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:56.170123 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:56.213454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce04af-b2db-48db-b7b1-431319ebf8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:56.214076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce04af-b2db-48db-b7b1-431319ebf8a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:56.214357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce04af-b2db-48db-b7b1-431319ebf8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:56.214628 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce04af-b2db-48db-b7b1-431319ebf8a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:56.215264 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce04af-b2db-48db-b7b1-431319ebf8a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:56.217298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce04af-b2db-48db-b7b1-431319ebf8a1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:56.240015 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1182/2371] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:03:56 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:03:56.244052 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b819e10-3791-47b7-af84-8e91389992d9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:56.254373 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b819e10-3791-47b7-af84-8e91389992d9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:56.258922 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fad96-c11e-441c-ba52-a6badbcf080c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:56.259511 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fad96-c11e-441c-ba52-a6badbcf080c admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:56.259786 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fad96-c11e-441c-ba52-a6badbcf080c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:56.260057 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fad96-c11e-441c-ba52-a6badbcf080c admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:56.260725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fad96-c11e-441c-ba52-a6badbcf080c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:56.262326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fad96-c11e-441c-ba52-a6badbcf080c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:56.272772 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1190/2372] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:03:56.277192 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce04af-b2db-48db-b7b1-431319ebf8a1 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:56.288287 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce04af-b2db-48db-b7b1-431319ebf8a1 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:56.293076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca409d8-eae6-47bf-b468-0abf3d089583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:56.293709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca409d8-eae6-47bf-b468-0abf3d089583 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:56.293991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca409d8-eae6-47bf-b468-0abf3d089583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:56.294265 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca409d8-eae6-47bf-b468-0abf3d089583 admin admin] PATH_INFO: `/v3/projects/6598d5e7c92a44538c4724274531b3fe/users/7e87c82bf0df40eb9bd9b6a8d27c030a/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:56.310864 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca409d8-eae6-47bf-b468-0abf3d089583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6598d5e7c92a44538c4724274531b3fe, user_id=7e87c82bf0df40eb9bd9b6a8d27c030a, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:56.313578 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca409d8-eae6-47bf-b468-0abf3d089583 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:56.327041 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1183/2373] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:03:56 2022] PUT /identity/v3/projects/6598d5e7c92a44538c4724274531b3fe/users/7e87c82bf0df40eb9bd9b6a8d27c030a/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:56.331307 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fad96-c11e-441c-ba52-a6badbcf080c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:56.341507 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fad96-c11e-441c-ba52-a6badbcf080c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:56.399105 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a2b45-cdf6-40dc-9a78-7e4409da131e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:56.400894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a2b45-cdf6-40dc-9a78-7e4409da131e admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:56.400894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a2b45-cdf6-40dc-9a78-7e4409da131e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:56.400894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a2b45-cdf6-40dc-9a78-7e4409da131e admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:56.401466 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a2b45-cdf6-40dc-9a78-7e4409da131e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:56.402786 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a2b45-cdf6-40dc-9a78-7e4409da131e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:56.405048 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:56.415887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1191/2374] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:03:56.420155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca409d8-eae6-47bf-b468-0abf3d089583 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:56.431239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca409d8-eae6-47bf-b468-0abf3d089583 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:56.435908 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc09bbc-122f-47a0-82aa-1216cc4c1040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:56.436713 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc09bbc-122f-47a0-82aa-1216cc4c1040 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:56.436995 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc09bbc-122f-47a0-82aa-1216cc4c1040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:56.437566 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc09bbc-122f-47a0-82aa-1216cc4c1040 admin admin] PATH_INFO: `/v3/projects/6598d5e7c92a44538c4724274531b3fe/users/7e87c82bf0df40eb9bd9b6a8d27c030a/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:56.439466 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc09bbc-122f-47a0-82aa-1216cc4c1040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6598d5e7c92a44538c4724274531b3fe, user_id=7e87c82bf0df40eb9bd9b6a8d27c030a, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:56.443903 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc09bbc-122f-47a0-82aa-1216cc4c1040 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:56.460209 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1184/2375] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:03:56 2022] PUT /identity/v3/projects/6598d5e7c92a44538c4724274531b3fe/users/7e87c82bf0df40eb9bd9b6a8d27c030a/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:56.469505 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c091144-f5ab-46ab-9059-88a2f0cc475c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:56.469814 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c091144-f5ab-46ab-9059-88a2f0cc475c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:56.470085 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c091144-f5ab-46ab-9059-88a2f0cc475c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:56.498823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3c091144-f5ab-46ab-9059-88a2f0cc475c None None] MFA Rules not processed for user `7e87c82bf0df40eb9bd9b6a8d27c030a`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:56.500021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3c091144-f5ab-46ab-9059-88a2f0cc475c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:56.567585 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1192/2376] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:56.578316 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc09bbc-122f-47a0-82aa-1216cc4c1040 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:56.580116 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2fc09bbc-122f-47a0-82aa-1216cc4c1040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:56.658729 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc09bbc-122f-47a0-82aa-1216cc4c1040 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:56.663066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80cf4d4-6011-48f5-94dc-07333e0deff4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:56.663670 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cf4d4-6011-48f5-94dc-07333e0deff4 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:56.663951 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cf4d4-6011-48f5-94dc-07333e0deff4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:56.664262 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cf4d4-6011-48f5-94dc-07333e0deff4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:56.673625 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80cf4d4-6011-48f5-94dc-07333e0deff4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:56.675151 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80cf4d4-6011-48f5-94dc-07333e0deff4 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:56.684820 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1185/2377] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:56 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:57.059724 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c091144-f5ab-46ab-9059-88a2f0cc475c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:57.121504 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c091144-f5ab-46ab-9059-88a2f0cc475c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:57.126829 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f18461-d8f1-49b9-bf8a-ef7f69807bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:57.127475 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f18461-d8f1-49b9-bf8a-ef7f69807bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:57.127783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f18461-d8f1-49b9-bf8a-ef7f69807bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:57.128770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f18461-d8f1-49b9-bf8a-ef7f69807bbe admin admin] PATH_INFO: `/v3/users/38b18b8deff0429a9743d56f72c7cd4e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:57.130299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f18461-d8f1-49b9-bf8a-ef7f69807bbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=38b18b8deff0429a9743d56f72c7cd4e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:57.133346 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f18461-d8f1-49b9-bf8a-ef7f69807bbe admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:57.189743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-80f18461-d8f1-49b9-bf8a-ef7f69807bbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38b18b8deff0429a9743d56f72c7cd4e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:57.190671 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-80f18461-d8f1-49b9-bf8a-ef7f69807bbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38b18b8deff0429a9743d56f72c7cd4e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:57.197625 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-80f18461-d8f1-49b9-bf8a-ef7f69807bbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38b18b8deff0429a9743d56f72c7cd4e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:57.204856 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-80f18461-d8f1-49b9-bf8a-ef7f69807bbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38b18b8deff0429a9743d56f72c7cd4e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:57.216897 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-80f18461-d8f1-49b9-bf8a-ef7f69807bbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38b18b8deff0429a9743d56f72c7cd4e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:57.223441 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1193/2378] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:03:57 2022] DELETE /identity/v3/users/38b18b8deff0429a9743d56f72c7cd4e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:57.227971 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1309d1-bf84-40d2-a44e-5f50f67ce453 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:57.228293 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1309d1-bf84-40d2-a44e-5f50f67ce453 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:57.228571 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1309d1-bf84-40d2-a44e-5f50f67ce453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:57.249701 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-ec1309d1-bf84-40d2-a44e-5f50f67ce453 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:57.250977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ec1309d1-bf84-40d2-a44e-5f50f67ce453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:57.299673 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:57.312585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1186/2379] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:57.322331 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f18461-d8f1-49b9-bf8a-ef7f69807bbe admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:57.323722 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-80f18461-d8f1-49b9-bf8a-ef7f69807bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:57.329155 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:57.384330 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f18461-d8f1-49b9-bf8a-ef7f69807bbe admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:57.388170 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e33167-2c9b-4308-8b31-a33775644cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:57.388784 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e33167-2c9b-4308-8b31-a33775644cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:57.389070 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e33167-2c9b-4308-8b31-a33775644cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:57.389347 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e33167-2c9b-4308-8b31-a33775644cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:57.399240 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e33167-2c9b-4308-8b31-a33775644cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:57.400713 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e33167-2c9b-4308-8b31-a33775644cc6 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:57.411282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1194/2380] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:57.683590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1309d1-bf84-40d2-a44e-5f50f67ce453 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:57.685102 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ec1309d1-bf84-40d2-a44e-5f50f67ce453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:57.697975 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1309d1-bf84-40d2-a44e-5f50f67ce453 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:57.703039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57402c7f-b1f9-47ee-973c-084f729ac0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:57.703676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57402c7f-b1f9-47ee-973c-084f729ac0d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:57.703957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57402c7f-b1f9-47ee-973c-084f729ac0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:57.704259 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57402c7f-b1f9-47ee-973c-084f729ac0d5 admin admin] PATH_INFO: `/v3/projects/e100437bdebd4b78ab0ddddc2c61ef8d` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:57.705404 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57402c7f-b1f9-47ee-973c-084f729ac0d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e100437bdebd4b78ab0ddddc2c61ef8d)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:57.708175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57402c7f-b1f9-47ee-973c-084f729ac0d5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:57.757673 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-57402c7f-b1f9-47ee-973c-084f729ac0d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e100437bdebd4b78ab0ddddc2c61ef8d', 'request_id': 'req-57402c7f-b1f9-47ee-973c-084f729ac0d5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:57.765033 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-57402c7f-b1f9-47ee-973c-084f729ac0d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e100437bdebd4b78ab0ddddc2c61ef8d', 'request_id': 'req-57402c7f-b1f9-47ee-973c-084f729ac0d5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:57.771387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-57402c7f-b1f9-47ee-973c-084f729ac0d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e100437bdebd4b78ab0ddddc2c61ef8d', 'request_id': 'req-57402c7f-b1f9-47ee-973c-084f729ac0d5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:57.782758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-57402c7f-b1f9-47ee-973c-084f729ac0d5 admin admin] The token cache is being invalidate because project e100437bdebd4b78ab0ddddc2c61ef8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:03:57.783046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-57402c7f-b1f9-47ee-973c-084f729ac0d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:57.783680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-57402c7f-b1f9-47ee-973c-084f729ac0d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:57.785663 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1187/2381] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:03:57 2022] DELETE /identity/v3/projects/e100437bdebd4b78ab0ddddc2c61ef8d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:57.789754 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e33167-2c9b-4308-8b31-a33775644cc6 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:57.799283 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e33167-2c9b-4308-8b31-a33775644cc6 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:57.855739 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f78f15e-b033-47ca-9b93-53f77e10c881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:57.856399 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78f15e-b033-47ca-9b93-53f77e10c881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:57.856681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78f15e-b033-47ca-9b93-53f77e10c881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:57.856956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78f15e-b033-47ca-9b93-53f77e10c881 admin admin] PATH_INFO: `/v3/users/7e87c82bf0df40eb9bd9b6a8d27c030a` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:57.858073 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f78f15e-b033-47ca-9b93-53f77e10c881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e87c82bf0df40eb9bd9b6a8d27c030a)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:57.859979 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f78f15e-b033-47ca-9b93-53f77e10c881 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:57.907362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9f78f15e-b033-47ca-9b93-53f77e10c881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e87c82bf0df40eb9bd9b6a8d27c030a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:57.908040 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9f78f15e-b033-47ca-9b93-53f77e10c881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e87c82bf0df40eb9bd9b6a8d27c030a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:57.915541 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9f78f15e-b033-47ca-9b93-53f77e10c881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e87c82bf0df40eb9bd9b6a8d27c030a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:57.922442 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9f78f15e-b033-47ca-9b93-53f77e10c881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e87c82bf0df40eb9bd9b6a8d27c030a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:57.934108 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9f78f15e-b033-47ca-9b93-53f77e10c881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e87c82bf0df40eb9bd9b6a8d27c030a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:57.939982 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1195/2382] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:03:57 2022] DELETE /identity/v3/users/7e87c82bf0df40eb9bd9b6a8d27c030a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:58.211071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57402c7f-b1f9-47ee-973c-084f729ac0d5 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:58.212684 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-57402c7f-b1f9-47ee-973c-084f729ac0d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:58.230005 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57402c7f-b1f9-47ee-973c-084f729ac0d5 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:58.288567 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e3466-2103-4131-af2a-61c89bf5123b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:58.289206 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e3466-2103-4131-af2a-61c89bf5123b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:58.289512 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e3466-2103-4131-af2a-61c89bf5123b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:58.289782 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e3466-2103-4131-af2a-61c89bf5123b admin admin] PATH_INFO: `/v3/projects/6598d5e7c92a44538c4724274531b3fe` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:58.290893 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e3466-2103-4131-af2a-61c89bf5123b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6598d5e7c92a44538c4724274531b3fe)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:58.293061 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e3466-2103-4131-af2a-61c89bf5123b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:58.342847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-790e3466-2103-4131-af2a-61c89bf5123b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6598d5e7c92a44538c4724274531b3fe', 'request_id': 'req-790e3466-2103-4131-af2a-61c89bf5123b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:58.350192 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-790e3466-2103-4131-af2a-61c89bf5123b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6598d5e7c92a44538c4724274531b3fe', 'request_id': 'req-790e3466-2103-4131-af2a-61c89bf5123b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:58.355703 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-790e3466-2103-4131-af2a-61c89bf5123b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6598d5e7c92a44538c4724274531b3fe', 'request_id': 'req-790e3466-2103-4131-af2a-61c89bf5123b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:58.367871 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-790e3466-2103-4131-af2a-61c89bf5123b admin admin] The token cache is being invalidate because project 6598d5e7c92a44538c4724274531b3fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:03:58.368327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-790e3466-2103-4131-af2a-61c89bf5123b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:58.369083 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-790e3466-2103-4131-af2a-61c89bf5123b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:58.370805 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1188/2383] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:03:58 2022] DELETE /identity/v3/projects/6598d5e7c92a44538c4724274531b3fe => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:58.375782 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d333ef4-b81d-4076-8f38-eeef70c954f7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:58.376313 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d333ef4-b81d-4076-8f38-eeef70c954f7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:58.376413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d333ef4-b81d-4076-8f38-eeef70c954f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:58.396431 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5d333ef4-b81d-4076-8f38-eeef70c954f7 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:58.397639 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5d333ef4-b81d-4076-8f38-eeef70c954f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:58.416363 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:58.459250 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1196/2384] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:58.468121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df57970-566e-48d5-a69e-613196f12193 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:58.468433 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df57970-566e-48d5-a69e-613196f12193 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:58.468707 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df57970-566e-48d5-a69e-613196f12193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:58.489336 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-6df57970-566e-48d5-a69e-613196f12193 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:58.490534 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6df57970-566e-48d5-a69e-613196f12193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:58.499880 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1189/2385] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:58.504177 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d333ef4-b81d-4076-8f38-eeef70c954f7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:58.513841 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d333ef4-b81d-4076-8f38-eeef70c954f7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:58.518367 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958beef0-2d19-4fe3-ada8-e770f96636c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:58.518991 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958beef0-2d19-4fe3-ada8-e770f96636c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:58.519280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958beef0-2d19-4fe3-ada8-e770f96636c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:58.519531 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958beef0-2d19-4fe3-ada8-e770f96636c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:58.520296 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958beef0-2d19-4fe3-ada8-e770f96636c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:03:58.520621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958beef0-2d19-4fe3-ada8-e770f96636c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:58.522236 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958beef0-2d19-4fe3-ada8-e770f96636c9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:58.534042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1197/2386] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:03:58.539006 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df57970-566e-48d5-a69e-613196f12193 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:58.543399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:58.550838 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df57970-566e-48d5-a69e-613196f12193 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:58.556654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1b5ae-282c-4116-8fe2-855b7674b68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:58.557254 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1b5ae-282c-4116-8fe2-855b7674b68b admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:58.557533 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1b5ae-282c-4116-8fe2-855b7674b68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:58.557805 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1b5ae-282c-4116-8fe2-855b7674b68b admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:58.558713 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1b5ae-282c-4116-8fe2-855b7674b68b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:58.560651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1b5ae-282c-4116-8fe2-855b7674b68b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:58.587797 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1190/2387] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:03:58 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:03:58.592355 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958beef0-2d19-4fe3-ada8-e770f96636c9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:58.602848 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958beef0-2d19-4fe3-ada8-e770f96636c9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:58.660362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20179ba2-abd8-4cf0-a741-d9347118d5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:58.661068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20179ba2-abd8-4cf0-a741-d9347118d5af admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:58.661349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20179ba2-abd8-4cf0-a741-d9347118d5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:58.661622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20179ba2-abd8-4cf0-a741-d9347118d5af admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:58.662259 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20179ba2-abd8-4cf0-a741-d9347118d5af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:58.664202 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20179ba2-abd8-4cf0-a741-d9347118d5af admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:58.686277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1198/2388] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:03:58 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:03:58.690434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1b5ae-282c-4116-8fe2-855b7674b68b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:58.701256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1b5ae-282c-4116-8fe2-855b7674b68b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:58.705977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf37d3e-31ef-495a-8392-e182ff6e5d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:58.706599 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf37d3e-31ef-495a-8392-e182ff6e5d60 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:58.706877 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf37d3e-31ef-495a-8392-e182ff6e5d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:58.707150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf37d3e-31ef-495a-8392-e182ff6e5d60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:58.707796 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf37d3e-31ef-495a-8392-e182ff6e5d60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:58.709474 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf37d3e-31ef-495a-8392-e182ff6e5d60 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:58.719571 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1191/2389] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:03:58.724505 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20179ba2-abd8-4cf0-a741-d9347118d5af admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:58.735360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20179ba2-abd8-4cf0-a741-d9347118d5af admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:58.740727 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e4d25-98c8-4c05-baab-833029c0813b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:58.741254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e4d25-98c8-4c05-baab-833029c0813b admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:58.741533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e4d25-98c8-4c05-baab-833029c0813b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:58.741797 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e4d25-98c8-4c05-baab-833029c0813b admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:58.742441 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e4d25-98c8-4c05-baab-833029c0813b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:58.744185 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e4d25-98c8-4c05-baab-833029c0813b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:58.754862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1199/2390] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:03:58.759125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf37d3e-31ef-495a-8392-e182ff6e5d60 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:58.769264 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf37d3e-31ef-495a-8392-e182ff6e5d60 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:58.773873 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f846841-1372-4cb9-99aa-07be2ba7d8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:58.774522 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f846841-1372-4cb9-99aa-07be2ba7d8c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:58.774808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f846841-1372-4cb9-99aa-07be2ba7d8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:58.775080 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f846841-1372-4cb9-99aa-07be2ba7d8c1 admin admin] PATH_INFO: `/v3/projects/a2faeb8cd2aa442290e04d3fc96f0d60/users/f03350a77ddb430686bb21e41e39ad4a/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:58.793791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f846841-1372-4cb9-99aa-07be2ba7d8c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2faeb8cd2aa442290e04d3fc96f0d60, user_id=f03350a77ddb430686bb21e41e39ad4a, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:58.796534 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f846841-1372-4cb9-99aa-07be2ba7d8c1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:58.815509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1192/2391] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:03:58 2022] PUT /identity/v3/projects/a2faeb8cd2aa442290e04d3fc96f0d60/users/f03350a77ddb430686bb21e41e39ad4a/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:58.826005 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8c089-1cd7-413f-90c1-945faf985d5a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:58.826293 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8c089-1cd7-413f-90c1-945faf985d5a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:58.826574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8c089-1cd7-413f-90c1-945faf985d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:58.854914 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-50c8c089-1cd7-413f-90c1-945faf985d5a None None] MFA Rules not processed for user `f03350a77ddb430686bb21e41e39ad4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:58.856166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-50c8c089-1cd7-413f-90c1-945faf985d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:58.922934 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1200/2392] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:58.934654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f846841-1372-4cb9-99aa-07be2ba7d8c1 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:58.936368 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3f846841-1372-4cb9-99aa-07be2ba7d8c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:58.998083 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f846841-1372-4cb9-99aa-07be2ba7d8c1 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:59.003707 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1316157-0d6e-4d7c-858e-13ec1e476773 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:59.004395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1316157-0d6e-4d7c-858e-13ec1e476773 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:59.004674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1316157-0d6e-4d7c-858e-13ec1e476773 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:59.004944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1316157-0d6e-4d7c-858e-13ec1e476773 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:59.014053 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1316157-0d6e-4d7c-858e-13ec1e476773 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:59.015480 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1316157-0d6e-4d7c-858e-13ec1e476773 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:59.025220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1193/2393] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:59.051394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8c089-1cd7-413f-90c1-945faf985d5a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:59.076811 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:59.125550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8c089-1cd7-413f-90c1-945faf985d5a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:59.131125 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721435ed-c2ac-49dd-babd-f7c824d20d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:59.131767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721435ed-c2ac-49dd-babd-f7c824d20d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:59.132050 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721435ed-c2ac-49dd-babd-f7c824d20d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:59.132357 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721435ed-c2ac-49dd-babd-f7c824d20d20 admin admin] PATH_INFO: `/v3/users/f03350a77ddb430686bb21e41e39ad4a` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:59.133504 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721435ed-c2ac-49dd-babd-f7c824d20d20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f03350a77ddb430686bb21e41e39ad4a)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:59.135426 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721435ed-c2ac-49dd-babd-f7c824d20d20 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:59.199966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-721435ed-c2ac-49dd-babd-f7c824d20d20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f03350a77ddb430686bb21e41e39ad4a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:59.201677 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-721435ed-c2ac-49dd-babd-f7c824d20d20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f03350a77ddb430686bb21e41e39ad4a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:59.209529 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-721435ed-c2ac-49dd-babd-f7c824d20d20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f03350a77ddb430686bb21e41e39ad4a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:59.217154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-721435ed-c2ac-49dd-babd-f7c824d20d20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f03350a77ddb430686bb21e41e39ad4a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:59.230080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-721435ed-c2ac-49dd-babd-f7c824d20d20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f03350a77ddb430686bb21e41e39ad4a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:59.236066 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1201/2394] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:03:59 2022] DELETE /identity/v3/users/f03350a77ddb430686bb21e41e39ad4a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:59.245919 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d5a55-f9c6-4760-8d9a-b71ebd306de0 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:59.246233 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d5a55-f9c6-4760-8d9a-b71ebd306de0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:59.246533 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d5a55-f9c6-4760-8d9a-b71ebd306de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:59.267663 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-8d9d5a55-f9c6-4760-8d9a-b71ebd306de0 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:59.269425 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8d9d5a55-f9c6-4760-8d9a-b71ebd306de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:59.327789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1194/2395] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:03:59.338134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721435ed-c2ac-49dd-babd-f7c824d20d20 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:59.339566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-721435ed-c2ac-49dd-babd-f7c824d20d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:59.397947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721435ed-c2ac-49dd-babd-f7c824d20d20 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:59.402011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316874d-a8f3-444c-9db4-e18fdc6609be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:59.402619 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316874d-a8f3-444c-9db4-e18fdc6609be service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:59.402904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316874d-a8f3-444c-9db4-e18fdc6609be service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:59.403175 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316874d-a8f3-444c-9db4-e18fdc6609be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:59.412336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316874d-a8f3-444c-9db4-e18fdc6609be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:59.413768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316874d-a8f3-444c-9db4-e18fdc6609be service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:59.423085 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1202/2396] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:03:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:03:59.703028 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d5a55-f9c6-4760-8d9a-b71ebd306de0 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:59.705166 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8d9d5a55-f9c6-4760-8d9a-b71ebd306de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:59.718590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d5a55-f9c6-4760-8d9a-b71ebd306de0 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:59.724313 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0b8e0-4dc0-4541-aa08-bda25ee22048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:59.724990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0b8e0-4dc0-4541-aa08-bda25ee22048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:59.725289 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0b8e0-4dc0-4541-aa08-bda25ee22048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:59.725562 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0b8e0-4dc0-4541-aa08-bda25ee22048 admin admin] PATH_INFO: `/v3/projects/a2faeb8cd2aa442290e04d3fc96f0d60` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:59.726708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0b8e0-4dc0-4541-aa08-bda25ee22048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2faeb8cd2aa442290e04d3fc96f0d60)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:59.728842 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0b8e0-4dc0-4541-aa08-bda25ee22048 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:59.774549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:59.778841 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2bc0b8e0-4dc0-4541-aa08-bda25ee22048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2faeb8cd2aa442290e04d3fc96f0d60', 'request_id': 'req-2bc0b8e0-4dc0-4541-aa08-bda25ee22048'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:59.787123 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2bc0b8e0-4dc0-4541-aa08-bda25ee22048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2faeb8cd2aa442290e04d3fc96f0d60', 'request_id': 'req-2bc0b8e0-4dc0-4541-aa08-bda25ee22048'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:59.793034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2bc0b8e0-4dc0-4541-aa08-bda25ee22048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2faeb8cd2aa442290e04d3fc96f0d60', 'request_id': 'req-2bc0b8e0-4dc0-4541-aa08-bda25ee22048'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:59.804325 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2bc0b8e0-4dc0-4541-aa08-bda25ee22048 admin admin] The token cache is being invalidate because project a2faeb8cd2aa442290e04d3fc96f0d60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:03:59.804614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2bc0b8e0-4dc0-4541-aa08-bda25ee22048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:59.805251 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2bc0b8e0-4dc0-4541-aa08-bda25ee22048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:03:59.806847 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1195/2397] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:03:59 2022] DELETE /identity/v3/projects/a2faeb8cd2aa442290e04d3fc96f0d60 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:03:59.811638 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f7743-01d6-4c03-b105-81e37fd18b64 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:59.811921 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f7743-01d6-4c03-b105-81e37fd18b64 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:59.812223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f7743-01d6-4c03-b105-81e37fd18b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:59.832004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-469f7743-01d6-4c03-b105-81e37fd18b64 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:59.833357 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-469f7743-01d6-4c03-b105-81e37fd18b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:59.872159 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:03:59.896285 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1203/2398] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:59.904442 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85893c53-80c9-4d51-8ac7-d6c5d782584d None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:59.904726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85893c53-80c9-4d51-8ac7-d6c5d782584d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:59.904998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85893c53-80c9-4d51-8ac7-d6c5d782584d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:59.925462 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-85893c53-80c9-4d51-8ac7-d6c5d782584d None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:03:59.926849 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-85893c53-80c9-4d51-8ac7-d6c5d782584d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:03:59.935503 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1196/2399] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:03:59.939716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f7743-01d6-4c03-b105-81e37fd18b64 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:59.949731 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f7743-01d6-4c03-b105-81e37fd18b64 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:59.954341 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b033ca1-869a-4e29-80e7-d28d2efe262d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:59.954949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b033ca1-869a-4e29-80e7-d28d2efe262d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:59.955225 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b033ca1-869a-4e29-80e7-d28d2efe262d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:59.955495 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b033ca1-869a-4e29-80e7-d28d2efe262d admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:59.956762 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b033ca1-869a-4e29-80e7-d28d2efe262d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:03:59.957072 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b033ca1-869a-4e29-80e7-d28d2efe262d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:59.958665 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b033ca1-869a-4e29-80e7-d28d2efe262d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:03:59.970300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1204/2400] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:03:59.975371 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85893c53-80c9-4d51-8ac7-d6c5d782584d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:03:59.984732 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85893c53-80c9-4d51-8ac7-d6c5d782584d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:03:59.989849 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be756e91-37f3-4f4a-a46c-9e3156842b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:03:59.990524 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be756e91-37f3-4f4a-a46c-9e3156842b1e admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:03:59.990809 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be756e91-37f3-4f4a-a46c-9e3156842b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:03:59.991083 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be756e91-37f3-4f4a-a46c-9e3156842b1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:03:59.991717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be756e91-37f3-4f4a-a46c-9e3156842b1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:03:59.993710 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be756e91-37f3-4f4a-a46c-9e3156842b1e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:00.021939 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1197/2401] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:03:59 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:00.026118 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b033ca1-869a-4e29-80e7-d28d2efe262d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:00.036606 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b033ca1-869a-4e29-80e7-d28d2efe262d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:00.096382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b692adf-1e67-4883-966d-fa122495dcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:00.096997 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b692adf-1e67-4883-966d-fa122495dcc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:00.097271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b692adf-1e67-4883-966d-fa122495dcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:00.097545 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b692adf-1e67-4883-966d-fa122495dcc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:00.098176 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b692adf-1e67-4883-966d-fa122495dcc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:00.100080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b692adf-1e67-4883-966d-fa122495dcc3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:00.122282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1205/2402] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:00 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:00.126486 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be756e91-37f3-4f4a-a46c-9e3156842b1e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:00.138954 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be756e91-37f3-4f4a-a46c-9e3156842b1e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:00.144682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8937b5-2253-4b68-aa8f-a9a5b73a90a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:00.145293 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8937b5-2253-4b68-aa8f-a9a5b73a90a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:00.145569 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8937b5-2253-4b68-aa8f-a9a5b73a90a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:00.145865 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8937b5-2253-4b68-aa8f-a9a5b73a90a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:00.146513 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8937b5-2253-4b68-aa8f-a9a5b73a90a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:00.148105 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8937b5-2253-4b68-aa8f-a9a5b73a90a1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:00.158663 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1198/2403] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:00.162639 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b692adf-1e67-4883-966d-fa122495dcc3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:00.172778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b692adf-1e67-4883-966d-fa122495dcc3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:00.177956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3311a-827d-4634-b4b3-f01e9924808d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:00.178556 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3311a-827d-4634-b4b3-f01e9924808d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:00.178835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3311a-827d-4634-b4b3-f01e9924808d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:00.179103 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3311a-827d-4634-b4b3-f01e9924808d admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:00.179744 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3311a-827d-4634-b4b3-f01e9924808d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:00.181356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3311a-827d-4634-b4b3-f01e9924808d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:00.192752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1206/2404] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:00.197642 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8937b5-2253-4b68-aa8f-a9a5b73a90a1 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:00.208489 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8937b5-2253-4b68-aa8f-a9a5b73a90a1 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:00.213342 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d80e1-aa72-4766-b7f1-a06c0c05704e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:00.213998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d80e1-aa72-4766-b7f1-a06c0c05704e admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:00.214279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d80e1-aa72-4766-b7f1-a06c0c05704e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:00.214552 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d80e1-aa72-4766-b7f1-a06c0c05704e admin admin] PATH_INFO: `/v3/projects/6fdd092b7ab44e2db4d4dd2029fa663a/users/0be8b1ed80344e0fa2782f5004cfd7a5/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:00.229856 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5d80e1-aa72-4766-b7f1-a06c0c05704e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fdd092b7ab44e2db4d4dd2029fa663a, user_id=0be8b1ed80344e0fa2782f5004cfd7a5, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:00.232541 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5d80e1-aa72-4766-b7f1-a06c0c05704e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:00.245808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1199/2405] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:00 2022] PUT /identity/v3/projects/6fdd092b7ab44e2db4d4dd2029fa663a/users/0be8b1ed80344e0fa2782f5004cfd7a5/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:00.257294 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8940925-b91c-4cbb-b87f-2c5a5ff4945c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:00.257584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8940925-b91c-4cbb-b87f-2c5a5ff4945c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:00.257863 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8940925-b91c-4cbb-b87f-2c5a5ff4945c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:00.292835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c8940925-b91c-4cbb-b87f-2c5a5ff4945c None None] MFA Rules not processed for user `0be8b1ed80344e0fa2782f5004cfd7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:00.294145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c8940925-b91c-4cbb-b87f-2c5a5ff4945c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:00.365455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1207/2406] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:04:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:00.370163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d80e1-aa72-4766-b7f1-a06c0c05704e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:00.442310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d80e1-aa72-4766-b7f1-a06c0c05704e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:00.446995 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f5f23-0da6-4daf-a10a-c493f05e80b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:00.447604 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f5f23-0da6-4daf-a10a-c493f05e80b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:00.447900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f5f23-0da6-4daf-a10a-c493f05e80b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:00.448387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f5f23-0da6-4daf-a10a-c493f05e80b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:00.448942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654f5f23-0da6-4daf-a10a-c493f05e80b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:00.450844 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654f5f23-0da6-4daf-a10a-c493f05e80b0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:00.479201 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1200/2407] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:00 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:04:00.482251 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8940925-b91c-4cbb-b87f-2c5a5ff4945c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:00.493642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8940925-b91c-4cbb-b87f-2c5a5ff4945c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:00.527633 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:00.551730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b091db-c8f4-46a9-b079-157a297aeb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:00.552299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b091db-c8f4-46a9-b079-157a297aeb0f admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:00.552606 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b091db-c8f4-46a9-b079-157a297aeb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:00.552879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b091db-c8f4-46a9-b079-157a297aeb0f admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:00.553507 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b091db-c8f4-46a9-b079-157a297aeb0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:00.555410 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b091db-c8f4-46a9-b079-157a297aeb0f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:00.583953 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1208/2408] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:00 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:00.585437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f5f23-0da6-4daf-a10a-c493f05e80b0 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:00.595708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f5f23-0da6-4daf-a10a-c493f05e80b0 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:00.600532 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b7cdd-9ef2-4a88-8918-861caaa4d152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:00.601129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b7cdd-9ef2-4a88-8918-861caaa4d152 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:00.601405 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b7cdd-9ef2-4a88-8918-861caaa4d152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:00.601675 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b7cdd-9ef2-4a88-8918-861caaa4d152 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:00.602320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b7cdd-9ef2-4a88-8918-861caaa4d152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:00.603910 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b7cdd-9ef2-4a88-8918-861caaa4d152 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:00.615037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1201/2409] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:00.619250 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b091db-c8f4-46a9-b079-157a297aeb0f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:00.630191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b091db-c8f4-46a9-b079-157a297aeb0f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:00.634938 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47307c72-cfdf-436d-a079-14caef0e1907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:00.635592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47307c72-cfdf-436d-a079-14caef0e1907 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:00.635874 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47307c72-cfdf-436d-a079-14caef0e1907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:00.636178 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47307c72-cfdf-436d-a079-14caef0e1907 admin admin] PATH_INFO: `/v3/projects/c61f5a447b154406a90d653050b10180/users/9d73df6d52e845eabe5428febabecb50/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:00.651145 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47307c72-cfdf-436d-a079-14caef0e1907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61f5a447b154406a90d653050b10180, user_id=9d73df6d52e845eabe5428febabecb50, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:00.654111 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47307c72-cfdf-436d-a079-14caef0e1907 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:00.666970 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1209/2410] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:00 2022] PUT /identity/v3/projects/c61f5a447b154406a90d653050b10180/users/9d73df6d52e845eabe5428febabecb50/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:00.670986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b7cdd-9ef2-4a88-8918-861caaa4d152 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:00.681541 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b7cdd-9ef2-4a88-8918-861caaa4d152 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:00.722455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:00.740842 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0065ee5-2e9d-4e88-9abd-a972a340b07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:00.741463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0065ee5-2e9d-4e88-9abd-a972a340b07a admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:00.741739 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0065ee5-2e9d-4e88-9abd-a972a340b07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:00.742014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0065ee5-2e9d-4e88-9abd-a972a340b07a admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:00.742691 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0065ee5-2e9d-4e88-9abd-a972a340b07a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:00.744401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0065ee5-2e9d-4e88-9abd-a972a340b07a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:00.754598 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1202/2411] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:00.758865 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47307c72-cfdf-436d-a079-14caef0e1907 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:00.769119 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47307c72-cfdf-436d-a079-14caef0e1907 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:00.774272 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec112cfa-f49b-4c54-a3fd-e1316a313767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:00.774956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec112cfa-f49b-4c54-a3fd-e1316a313767 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:00.775234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec112cfa-f49b-4c54-a3fd-e1316a313767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:00.775505 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec112cfa-f49b-4c54-a3fd-e1316a313767 admin admin] PATH_INFO: `/v3/projects/c61f5a447b154406a90d653050b10180/users/9d73df6d52e845eabe5428febabecb50/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:00.777710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec112cfa-f49b-4c54-a3fd-e1316a313767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61f5a447b154406a90d653050b10180, user_id=9d73df6d52e845eabe5428febabecb50, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:00.780379 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec112cfa-f49b-4c54-a3fd-e1316a313767 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:00.791366 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1210/2412] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:00 2022] PUT /identity/v3/projects/c61f5a447b154406a90d653050b10180/users/9d73df6d52e845eabe5428febabecb50/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:00.801992 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6a767-9606-4e70-aa4b-9ffc6d825649 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:00.802238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6a767-9606-4e70-aa4b-9ffc6d825649 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:00.802514 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6a767-9606-4e70-aa4b-9ffc6d825649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:00.829848 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-57d6a767-9606-4e70-aa4b-9ffc6d825649 None None] MFA Rules not processed for user `9d73df6d52e845eabe5428febabecb50`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:00.831054 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-57d6a767-9606-4e70-aa4b-9ffc6d825649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:00.899408 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1203/2413] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:04:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:00.911095 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec112cfa-f49b-4c54-a3fd-e1316a313767 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:00.913050 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ec112cfa-f49b-4c54-a3fd-e1316a313767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:00.970796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec112cfa-f49b-4c54-a3fd-e1316a313767 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:00.975442 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52869726-3bc4-4704-9cda-a125234be124 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:00.976057 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52869726-3bc4-4704-9cda-a125234be124 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:00.976379 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52869726-3bc4-4704-9cda-a125234be124 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:00.976657 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52869726-3bc4-4704-9cda-a125234be124 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:01.033214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52869726-3bc4-4704-9cda-a125234be124 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:01.035200 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52869726-3bc4-4704-9cda-a125234be124 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:01.044046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1211/2414] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:04:03.586213 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6a767-9606-4e70-aa4b-9ffc6d825649 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:03.648520 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6a767-9606-4e70-aa4b-9ffc6d825649 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:03.653429 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba0b78-eb86-4061-b523-f3df614a24a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:03.654065 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba0b78-eb86-4061-b523-f3df614a24a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:03.654346 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba0b78-eb86-4061-b523-f3df614a24a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:03.654619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba0b78-eb86-4061-b523-f3df614a24a7 admin admin] PATH_INFO: `/v3/users/0be8b1ed80344e0fa2782f5004cfd7a5` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:03.655747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba0b78-eb86-4061-b523-f3df614a24a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0be8b1ed80344e0fa2782f5004cfd7a5)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:03.657709 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba0b78-eb86-4061-b523-f3df614a24a7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:03.706441 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-26ba0b78-eb86-4061-b523-f3df614a24a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0be8b1ed80344e0fa2782f5004cfd7a5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:03.714150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-26ba0b78-eb86-4061-b523-f3df614a24a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0be8b1ed80344e0fa2782f5004cfd7a5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:03.726194 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:03.731228 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-26ba0b78-eb86-4061-b523-f3df614a24a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0be8b1ed80344e0fa2782f5004cfd7a5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:03.739233 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-26ba0b78-eb86-4061-b523-f3df614a24a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0be8b1ed80344e0fa2782f5004cfd7a5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:03.740004 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-26ba0b78-eb86-4061-b523-f3df614a24a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0be8b1ed80344e0fa2782f5004cfd7a5'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:03.745990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1204/2415] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:03 2022] DELETE /identity/v3/users/0be8b1ed80344e0fa2782f5004cfd7a5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:03.750439 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0123ca-f318-4038-bdfa-f4df20ddbfe4 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:03.750910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0123ca-f318-4038-bdfa-f4df20ddbfe4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:03.751036 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0123ca-f318-4038-bdfa-f4df20ddbfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:03.770597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-fd0123ca-f318-4038-bdfa-f4df20ddbfe4 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:03.771797 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fd0123ca-f318-4038-bdfa-f4df20ddbfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:03.814247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:03.835207 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1212/2416] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:03.844952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba0b78-eb86-4061-b523-f3df614a24a7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:03.849225 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-26ba0b78-eb86-4061-b523-f3df614a24a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:03.910616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba0b78-eb86-4061-b523-f3df614a24a7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:03.914789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f79542-4960-44d2-b951-9d88a95a1efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:03.915577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f79542-4960-44d2-b951-9d88a95a1efd service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:03.915892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f79542-4960-44d2-b951-9d88a95a1efd service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:03.916206 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f79542-4960-44d2-b951-9d88a95a1efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:03.928560 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f79542-4960-44d2-b951-9d88a95a1efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:03.930376 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f79542-4960-44d2-b951-9d88a95a1efd service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:03.941398 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1205/2417] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:04.231996 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0123ca-f318-4038-bdfa-f4df20ddbfe4 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:04.233461 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fd0123ca-f318-4038-bdfa-f4df20ddbfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:04.246118 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0123ca-f318-4038-bdfa-f4df20ddbfe4 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:04.250830 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15548e-8a49-42bf-9c0d-c228a8b45d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:04.251516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15548e-8a49-42bf-9c0d-c228a8b45d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:04.251802 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15548e-8a49-42bf-9c0d-c228a8b45d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:04.252078 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15548e-8a49-42bf-9c0d-c228a8b45d3a admin admin] PATH_INFO: `/v3/projects/6fdd092b7ab44e2db4d4dd2029fa663a` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:04.253325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15548e-8a49-42bf-9c0d-c228a8b45d3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fdd092b7ab44e2db4d4dd2029fa663a)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:04.256186 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15548e-8a49-42bf-9c0d-c228a8b45d3a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:04.304552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5e15548e-8a49-42bf-9c0d-c228a8b45d3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fdd092b7ab44e2db4d4dd2029fa663a', 'request_id': 'req-5e15548e-8a49-42bf-9c0d-c228a8b45d3a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:04.309985 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5e15548e-8a49-42bf-9c0d-c228a8b45d3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fdd092b7ab44e2db4d4dd2029fa663a', 'request_id': 'req-5e15548e-8a49-42bf-9c0d-c228a8b45d3a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:04.317654 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5e15548e-8a49-42bf-9c0d-c228a8b45d3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fdd092b7ab44e2db4d4dd2029fa663a', 'request_id': 'req-5e15548e-8a49-42bf-9c0d-c228a8b45d3a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:04.328692 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5e15548e-8a49-42bf-9c0d-c228a8b45d3a admin admin] The token cache is being invalidate because project 6fdd092b7ab44e2db4d4dd2029fa663a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:04.328982 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5e15548e-8a49-42bf-9c0d-c228a8b45d3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:04.329629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5e15548e-8a49-42bf-9c0d-c228a8b45d3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:04.331137 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1213/2418] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:04 2022] DELETE /identity/v3/projects/6fdd092b7ab44e2db4d4dd2029fa663a => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:04.335323 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f79542-4960-44d2-b951-9d88a95a1efd service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:04.344916 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f79542-4960-44d2-b951-9d88a95a1efd service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:04.402168 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc5ccb-0e88-4121-b3bf-b411e6278c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:04.402794 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc5ccb-0e88-4121-b3bf-b411e6278c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:04.403105 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc5ccb-0e88-4121-b3bf-b411e6278c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:04.403378 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc5ccb-0e88-4121-b3bf-b411e6278c3d admin admin] PATH_INFO: `/v3/users/9d73df6d52e845eabe5428febabecb50` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:04.404526 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc5ccb-0e88-4121-b3bf-b411e6278c3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d73df6d52e845eabe5428febabecb50)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:04.406475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc5ccb-0e88-4121-b3bf-b411e6278c3d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:04.452381 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-88bc5ccb-0e88-4121-b3bf-b411e6278c3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d73df6d52e845eabe5428febabecb50'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:04.461858 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-88bc5ccb-0e88-4121-b3bf-b411e6278c3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d73df6d52e845eabe5428febabecb50'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:04.474826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-88bc5ccb-0e88-4121-b3bf-b411e6278c3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d73df6d52e845eabe5428febabecb50'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:04.481736 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-88bc5ccb-0e88-4121-b3bf-b411e6278c3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d73df6d52e845eabe5428febabecb50'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:04.482429 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-88bc5ccb-0e88-4121-b3bf-b411e6278c3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d73df6d52e845eabe5428febabecb50'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:04.488336 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1206/2419] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:04 2022] DELETE /identity/v3/users/9d73df6d52e845eabe5428febabecb50 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:04.763352 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15548e-8a49-42bf-9c0d-c228a8b45d3a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:04.764964 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5e15548e-8a49-42bf-9c0d-c228a8b45d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:04.779450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15548e-8a49-42bf-9c0d-c228a8b45d3a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:04.839051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea278d-da78-46c3-a8ac-f857497dc403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:04.839692 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea278d-da78-46c3-a8ac-f857497dc403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:04.839977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea278d-da78-46c3-a8ac-f857497dc403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:04.840283 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea278d-da78-46c3-a8ac-f857497dc403 admin admin] PATH_INFO: `/v3/projects/c61f5a447b154406a90d653050b10180` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:04.841396 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea278d-da78-46c3-a8ac-f857497dc403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c61f5a447b154406a90d653050b10180)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:04.843298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea278d-da78-46c3-a8ac-f857497dc403 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:04.890027 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f9ea278d-da78-46c3-a8ac-f857497dc403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c61f5a447b154406a90d653050b10180', 'request_id': 'req-f9ea278d-da78-46c3-a8ac-f857497dc403'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:04.896178 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f9ea278d-da78-46c3-a8ac-f857497dc403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c61f5a447b154406a90d653050b10180', 'request_id': 'req-f9ea278d-da78-46c3-a8ac-f857497dc403'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:04.903704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f9ea278d-da78-46c3-a8ac-f857497dc403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c61f5a447b154406a90d653050b10180', 'request_id': 'req-f9ea278d-da78-46c3-a8ac-f857497dc403'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:04.911524 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:04.916628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f9ea278d-da78-46c3-a8ac-f857497dc403 admin admin] The token cache is being invalidate because project c61f5a447b154406a90d653050b10180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:04.917074 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f9ea278d-da78-46c3-a8ac-f857497dc403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:04.918216 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f9ea278d-da78-46c3-a8ac-f857497dc403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:04.919820 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1214/2420] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:04 2022] DELETE /identity/v3/projects/c61f5a447b154406a90d653050b10180 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:04.924812 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f1be3-3520-44b6-a638-958f8431bd53 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:04.925286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f1be3-3520-44b6-a638-958f8431bd53 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:04.925382 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f1be3-3520-44b6-a638-958f8431bd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:04.934046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:04.946244 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-cb7f1be3-3520-44b6-a638-958f8431bd53 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:04.947456 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-cb7f1be3-3520-44b6-a638-958f8431bd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:05.011642 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1207/2421] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:05.021025 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17bb89-de83-4f8d-8de9-8c572e7d34c8 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:05.021409 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17bb89-de83-4f8d-8de9-8c572e7d34c8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:05.021688 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17bb89-de83-4f8d-8de9-8c572e7d34c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:05.041458 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5c17bb89-de83-4f8d-8de9-8c572e7d34c8 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:05.042741 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5c17bb89-de83-4f8d-8de9-8c572e7d34c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:05.050935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1215/2422] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:05.055906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f1be3-3520-44b6-a638-958f8431bd53 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:05.068663 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f1be3-3520-44b6-a638-958f8431bd53 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:05.073259 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54fce68-5b30-4a07-8709-92d6a01d6b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:05.073875 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fce68-5b30-4a07-8709-92d6a01d6b35 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:05.074187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fce68-5b30-4a07-8709-92d6a01d6b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:05.074459 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fce68-5b30-4a07-8709-92d6a01d6b35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:05.075115 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54fce68-5b30-4a07-8709-92d6a01d6b35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:04:05.075417 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54fce68-5b30-4a07-8709-92d6a01d6b35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:05.077083 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54fce68-5b30-4a07-8709-92d6a01d6b35 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:05.090726 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1208/2423] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:04:05.095105 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17bb89-de83-4f8d-8de9-8c572e7d34c8 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:05.105112 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17bb89-de83-4f8d-8de9-8c572e7d34c8 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:05.109853 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a645e-569f-47c5-84a7-784a6f987c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:05.110464 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a645e-569f-47c5-84a7-784a6f987c49 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:05.110745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a645e-569f-47c5-84a7-784a6f987c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:05.111019 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a645e-569f-47c5-84a7-784a6f987c49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:05.111660 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a645e-569f-47c5-84a7-784a6f987c49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:05.114167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a645e-569f-47c5-84a7-784a6f987c49 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:05.145896 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1216/2424] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:05 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:05.149884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54fce68-5b30-4a07-8709-92d6a01d6b35 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:05.160075 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54fce68-5b30-4a07-8709-92d6a01d6b35 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:05.218617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01aebe-e1dd-491b-b155-3707695a9dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:05.219229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01aebe-e1dd-491b-b155-3707695a9dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:05.219518 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01aebe-e1dd-491b-b155-3707695a9dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:05.219791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01aebe-e1dd-491b-b155-3707695a9dc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:05.220441 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01aebe-e1dd-491b-b155-3707695a9dc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:05.222357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01aebe-e1dd-491b-b155-3707695a9dc6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:05.248155 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1209/2425] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:05 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:05.249551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a645e-569f-47c5-84a7-784a6f987c49 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:05.260038 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a645e-569f-47c5-84a7-784a6f987c49 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:05.264605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508833a9-0eee-4339-9120-e5bdffafd42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:05.265223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508833a9-0eee-4339-9120-e5bdffafd42c admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:05.265497 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508833a9-0eee-4339-9120-e5bdffafd42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:05.265760 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508833a9-0eee-4339-9120-e5bdffafd42c admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:05.266484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508833a9-0eee-4339-9120-e5bdffafd42c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:05.267992 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508833a9-0eee-4339-9120-e5bdffafd42c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:05.278669 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1217/2426] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:05.289956 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01aebe-e1dd-491b-b155-3707695a9dc6 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:05.300321 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01aebe-e1dd-491b-b155-3707695a9dc6 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:05.305628 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4f5a4-e486-474d-a8ab-03930aab03d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:05.306226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4f5a4-e486-474d-a8ab-03930aab03d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:05.306497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4f5a4-e486-474d-a8ab-03930aab03d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:05.306764 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4f5a4-e486-474d-a8ab-03930aab03d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:05.307401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4f5a4-e486-474d-a8ab-03930aab03d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:05.309030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4f5a4-e486-474d-a8ab-03930aab03d3 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:05.320000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1210/2427] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:05.324422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508833a9-0eee-4339-9120-e5bdffafd42c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:05.334779 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508833a9-0eee-4339-9120-e5bdffafd42c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:05.339335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ce127-b620-41a7-b563-61d22cc1ef15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:05.339981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ce127-b620-41a7-b563-61d22cc1ef15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:05.340294 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ce127-b620-41a7-b563-61d22cc1ef15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:05.340567 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ce127-b620-41a7-b563-61d22cc1ef15 admin admin] PATH_INFO: `/v3/projects/9db2ed6b104f4f5887cdeba0da6cf3ec/users/32f50eb99f1046529f81cc2c91a5f879/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:05.355737 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ce127-b620-41a7-b563-61d22cc1ef15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db2ed6b104f4f5887cdeba0da6cf3ec, user_id=32f50eb99f1046529f81cc2c91a5f879, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:05.358448 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ce127-b620-41a7-b563-61d22cc1ef15 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:05.373114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1218/2428] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:05 2022] PUT /identity/v3/projects/9db2ed6b104f4f5887cdeba0da6cf3ec/users/32f50eb99f1046529f81cc2c91a5f879/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:05.439546 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9b26c-8ecb-4e7c-8a14-6fb2579086d7 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:05.439777 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9b26c-8ecb-4e7c-8a14-6fb2579086d7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:05.440040 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9b26c-8ecb-4e7c-8a14-6fb2579086d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:05.470001 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1ff9b26c-8ecb-4e7c-8a14-6fb2579086d7 None None] MFA Rules not processed for user `32f50eb99f1046529f81cc2c91a5f879`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:05.471241 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1ff9b26c-8ecb-4e7c-8a14-6fb2579086d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:05.532347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:05.536691 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1211/2429] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:04:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:05.540989 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ce127-b620-41a7-b563-61d22cc1ef15 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:05.605058 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ce127-b620-41a7-b563-61d22cc1ef15 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:05.609774 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820b856-aa44-4ef5-98d3-bba9f7b31ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:05.610413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820b856-aa44-4ef5-98d3-bba9f7b31ed8 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:05.610814 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820b856-aa44-4ef5-98d3-bba9f7b31ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:05.611093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820b856-aa44-4ef5-98d3-bba9f7b31ed8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:05.611730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820b856-aa44-4ef5-98d3-bba9f7b31ed8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:05.614441 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820b856-aa44-4ef5-98d3-bba9f7b31ed8 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:05.641007 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1219/2430] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:05 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:05.645324 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9b26c-8ecb-4e7c-8a14-6fb2579086d7 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:05.656392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9b26c-8ecb-4e7c-8a14-6fb2579086d7 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:05.712655 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c165511-c1b6-4fd4-b54c-4cddd47c80aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:05.713189 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c165511-c1b6-4fd4-b54c-4cddd47c80aa admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:05.713472 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c165511-c1b6-4fd4-b54c-4cddd47c80aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:05.713746 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c165511-c1b6-4fd4-b54c-4cddd47c80aa admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:05.714383 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c165511-c1b6-4fd4-b54c-4cddd47c80aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:05.716407 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c165511-c1b6-4fd4-b54c-4cddd47c80aa admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:05.741570 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1212/2431] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:05 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:05.746101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820b856-aa44-4ef5-98d3-bba9f7b31ed8 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:05.756595 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820b856-aa44-4ef5-98d3-bba9f7b31ed8 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:05.761226 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4f996-a3de-49d7-99b4-3376997224a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:05.761828 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4f996-a3de-49d7-99b4-3376997224a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:05.762117 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4f996-a3de-49d7-99b4-3376997224a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:05.762407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4f996-a3de-49d7-99b4-3376997224a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:05.763048 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4f996-a3de-49d7-99b4-3376997224a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:05.764679 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4f996-a3de-49d7-99b4-3376997224a3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:05.775096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1220/2432] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:05.779340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c165511-c1b6-4fd4-b54c-4cddd47c80aa admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:05.789593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c165511-c1b6-4fd4-b54c-4cddd47c80aa admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:05.794228 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b86cd3-92ba-4cc3-8cab-4abdd1491c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:05.794790 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b86cd3-92ba-4cc3-8cab-4abdd1491c08 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:05.795067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b86cd3-92ba-4cc3-8cab-4abdd1491c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:05.795334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b86cd3-92ba-4cc3-8cab-4abdd1491c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:05.795966 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b86cd3-92ba-4cc3-8cab-4abdd1491c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:05.797590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b86cd3-92ba-4cc3-8cab-4abdd1491c08 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:05.808016 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1213/2433] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:05.812279 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4f996-a3de-49d7-99b4-3376997224a3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:05.822995 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4f996-a3de-49d7-99b4-3376997224a3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:05.827660 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cb2bc-84ea-493b-9ba4-3183035226a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:05.828339 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cb2bc-84ea-493b-9ba4-3183035226a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:05.828623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cb2bc-84ea-493b-9ba4-3183035226a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:05.828894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cb2bc-84ea-493b-9ba4-3183035226a9 admin admin] PATH_INFO: `/v3/projects/ee58336bb8164d06a9ccdd29d1ec293e/users/eecee98a53354225b8c7acae5cfd13ac/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:05.843558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2cb2bc-84ea-493b-9ba4-3183035226a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee58336bb8164d06a9ccdd29d1ec293e, user_id=eecee98a53354225b8c7acae5cfd13ac, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:05.846245 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2cb2bc-84ea-493b-9ba4-3183035226a9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:05.859135 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1221/2434] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:05 2022] PUT /identity/v3/projects/ee58336bb8164d06a9ccdd29d1ec293e/users/eecee98a53354225b8c7acae5cfd13ac/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:05.869772 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a2737-9320-49ca-9afd-1e2a76f2c60d None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:05.870055 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a2737-9320-49ca-9afd-1e2a76f2c60d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:05.870327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a2737-9320-49ca-9afd-1e2a76f2c60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:05.898169 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-764a2737-9320-49ca-9afd-1e2a76f2c60d None None] MFA Rules not processed for user `eecee98a53354225b8c7acae5cfd13ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:05.899358 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-764a2737-9320-49ca-9afd-1e2a76f2c60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:05.963750 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1214/2435] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:04:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:05.968330 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cb2bc-84ea-493b-9ba4-3183035226a9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:06.017680 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:06.036556 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cb2bc-84ea-493b-9ba4-3183035226a9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:06.041155 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0819ec-9ac0-48c2-bfb2-fb752db899dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:06.041761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0819ec-9ac0-48c2-bfb2-fb752db899dc admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:06.042072 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0819ec-9ac0-48c2-bfb2-fb752db899dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:06.042343 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0819ec-9ac0-48c2-bfb2-fb752db899dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:06.043009 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0819ec-9ac0-48c2-bfb2-fb752db899dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:06.044887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0819ec-9ac0-48c2-bfb2-fb752db899dc admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:06.071987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1222/2436] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:05 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:06.076123 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a2737-9320-49ca-9afd-1e2a76f2c60d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:06.086652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a2737-9320-49ca-9afd-1e2a76f2c60d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:06.143440 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339ac47-90c7-4388-a9a1-0a3996f20ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:06.144051 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339ac47-90c7-4388-a9a1-0a3996f20ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:06.144373 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339ac47-90c7-4388-a9a1-0a3996f20ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:06.144712 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339ac47-90c7-4388-a9a1-0a3996f20ed7 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:06.145820 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339ac47-90c7-4388-a9a1-0a3996f20ed7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:06.147416 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339ac47-90c7-4388-a9a1-0a3996f20ed7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:06.167017 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:06.179754 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1215/2437] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:06 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:06.181371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0819ec-9ac0-48c2-bfb2-fb752db899dc admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:06.192515 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0819ec-9ac0-48c2-bfb2-fb752db899dc admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:06.197402 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c766a-7ea0-4a9f-8bca-4edb0a637727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:06.198008 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c766a-7ea0-4a9f-8bca-4edb0a637727 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:06.198285 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c766a-7ea0-4a9f-8bca-4edb0a637727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:06.198552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c766a-7ea0-4a9f-8bca-4edb0a637727 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:06.199206 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c766a-7ea0-4a9f-8bca-4edb0a637727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:06.200837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c766a-7ea0-4a9f-8bca-4edb0a637727 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:06.214411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1223/2438] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:04:06.218020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339ac47-90c7-4388-a9a1-0a3996f20ed7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:06.228326 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339ac47-90c7-4388-a9a1-0a3996f20ed7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:06.233262 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfade5a0-e6b8-4690-be56-3f81c75ae6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:06.233928 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfade5a0-e6b8-4690-be56-3f81c75ae6f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:06.234207 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfade5a0-e6b8-4690-be56-3f81c75ae6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:06.234481 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfade5a0-e6b8-4690-be56-3f81c75ae6f0 admin admin] PATH_INFO: `/v3/projects/8ec243bacf9447bc897ada70d2474d11/users/119f48ecfb2f4da3a13ff5ad3ff8e141/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:06.251825 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfade5a0-e6b8-4690-be56-3f81c75ae6f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ec243bacf9447bc897ada70d2474d11, user_id=119f48ecfb2f4da3a13ff5ad3ff8e141, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:06.255063 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfade5a0-e6b8-4690-be56-3f81c75ae6f0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:06.268810 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1216/2439] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:06 2022] PUT /identity/v3/projects/8ec243bacf9447bc897ada70d2474d11/users/119f48ecfb2f4da3a13ff5ad3ff8e141/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:06.273397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c766a-7ea0-4a9f-8bca-4edb0a637727 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:06.283791 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c766a-7ea0-4a9f-8bca-4edb0a637727 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:06.346771 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13485f52-4027-4705-91be-1531b16e9a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:06.347387 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13485f52-4027-4705-91be-1531b16e9a4c admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:06.347668 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13485f52-4027-4705-91be-1531b16e9a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:06.347945 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13485f52-4027-4705-91be-1531b16e9a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:06.349294 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13485f52-4027-4705-91be-1531b16e9a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:06.350865 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13485f52-4027-4705-91be-1531b16e9a4c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:06.360590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1224/2440] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:06.365000 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfade5a0-e6b8-4690-be56-3f81c75ae6f0 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:06.375121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfade5a0-e6b8-4690-be56-3f81c75ae6f0 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:06.379721 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44eee8-bf8b-4ba2-8ac2-d64892704328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:06.380419 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44eee8-bf8b-4ba2-8ac2-d64892704328 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:06.380678 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44eee8-bf8b-4ba2-8ac2-d64892704328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:06.380949 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44eee8-bf8b-4ba2-8ac2-d64892704328 admin admin] PATH_INFO: `/v3/projects/8ec243bacf9447bc897ada70d2474d11/users/119f48ecfb2f4da3a13ff5ad3ff8e141/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:06.382831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44eee8-bf8b-4ba2-8ac2-d64892704328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ec243bacf9447bc897ada70d2474d11, user_id=119f48ecfb2f4da3a13ff5ad3ff8e141, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:06.385492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44eee8-bf8b-4ba2-8ac2-d64892704328 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:06.396273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1217/2441] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:06 2022] PUT /identity/v3/projects/8ec243bacf9447bc897ada70d2474d11/users/119f48ecfb2f4da3a13ff5ad3ff8e141/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:06.408783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25360cfe-bd3b-4a04-b043-e76891eb9324 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:06.409278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25360cfe-bd3b-4a04-b043-e76891eb9324 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:06.409752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25360cfe-bd3b-4a04-b043-e76891eb9324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:06.439093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-25360cfe-bd3b-4a04-b043-e76891eb9324 None None] MFA Rules not processed for user `119f48ecfb2f4da3a13ff5ad3ff8e141`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:06.440352 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-25360cfe-bd3b-4a04-b043-e76891eb9324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:06.509303 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1225/2442] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:04:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:06.521011 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44eee8-bf8b-4ba2-8ac2-d64892704328 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:06.522610 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2f44eee8-bf8b-4ba2-8ac2-d64892704328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:06.581105 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44eee8-bf8b-4ba2-8ac2-d64892704328 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:06.585369 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c850705-4964-4c0b-aa9f-e2310b4ec74a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:06.585973 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c850705-4964-4c0b-aa9f-e2310b4ec74a service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:06.586250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c850705-4964-4c0b-aa9f-e2310b4ec74a service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:06.586519 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c850705-4964-4c0b-aa9f-e2310b4ec74a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:06.641490 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c850705-4964-4c0b-aa9f-e2310b4ec74a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:06.642928 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c850705-4964-4c0b-aa9f-e2310b4ec74a service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:06.652571 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1218/2443] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:08.162949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25360cfe-bd3b-4a04-b043-e76891eb9324 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:08.164502 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-25360cfe-bd3b-4a04-b043-e76891eb9324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:08.176030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25360cfe-bd3b-4a04-b043-e76891eb9324 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:08.182761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47597785-fe9b-4871-9c8a-65a78e4cd490 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:08.183417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47597785-fe9b-4871-9c8a-65a78e4cd490 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:08.183666 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47597785-fe9b-4871-9c8a-65a78e4cd490 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:08.183942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47597785-fe9b-4871-9c8a-65a78e4cd490 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:08.228371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:08.248165 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47597785-fe9b-4871-9c8a-65a78e4cd490 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:08.249640 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47597785-fe9b-4871-9c8a-65a78e4cd490 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:08.258622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1226/2444] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:08.799118 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c850705-4964-4c0b-aa9f-e2310b4ec74a service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:08.807972 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c850705-4964-4c0b-aa9f-e2310b4ec74a service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:08.811555 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f417c-8a0c-4840-9c15-d804da695fc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:08.812196 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f417c-8a0c-4840-9c15-d804da695fc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:08.812453 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f417c-8a0c-4840-9c15-d804da695fc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:08.812721 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f417c-8a0c-4840-9c15-d804da695fc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:08.821880 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f417c-8a0c-4840-9c15-d804da695fc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:08.823282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f417c-8a0c-4840-9c15-d804da695fc6 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:08.833179 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1219/2445] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:08 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:11.659497 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47597785-fe9b-4871-9c8a-65a78e4cd490 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:11.661028 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-47597785-fe9b-4871-9c8a-65a78e4cd490 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:11.757240 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47597785-fe9b-4871-9c8a-65a78e4cd490 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:11.763501 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620675d1-24e1-497c-a1b4-0be3f2c88d7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33a0844a1c8f4ea4932e12d53e5ee9ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:11.764358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620675d1-24e1-497c-a1b4-0be3f2c88d7a service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:11.764636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620675d1-24e1-497c-a1b4-0be3f2c88d7a service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:11.764901 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620675d1-24e1-497c-a1b4-0be3f2c88d7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:11.766635 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-620675d1-24e1-497c-a1b4-0be3f2c88d7a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:11.778227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620675d1-24e1-497c-a1b4-0be3f2c88d7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:11.779746 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620675d1-24e1-497c-a1b4-0be3f2c88d7a service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:11.791318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1227/2446] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:12.725718 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f417c-8a0c-4840-9c15-d804da695fc6 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:12.793369 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f417c-8a0c-4840-9c15-d804da695fc6 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:12.797895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54490a46-de67-4abe-a67c-144ab46233bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:12.798525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54490a46-de67-4abe-a67c-144ab46233bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:12.798802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54490a46-de67-4abe-a67c-144ab46233bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:12.799071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54490a46-de67-4abe-a67c-144ab46233bb admin admin] PATH_INFO: `/v3/users/32f50eb99f1046529f81cc2c91a5f879` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:12.800219 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54490a46-de67-4abe-a67c-144ab46233bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=32f50eb99f1046529f81cc2c91a5f879)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:12.802201 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54490a46-de67-4abe-a67c-144ab46233bb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:12.804549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:12.860481 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-54490a46-de67-4abe-a67c-144ab46233bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32f50eb99f1046529f81cc2c91a5f879'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:12.868920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-54490a46-de67-4abe-a67c-144ab46233bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32f50eb99f1046529f81cc2c91a5f879'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:12.883277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-54490a46-de67-4abe-a67c-144ab46233bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32f50eb99f1046529f81cc2c91a5f879'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:12.891609 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-54490a46-de67-4abe-a67c-144ab46233bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32f50eb99f1046529f81cc2c91a5f879'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:12.892406 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-54490a46-de67-4abe-a67c-144ab46233bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32f50eb99f1046529f81cc2c91a5f879'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:12.907579 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1220/2447] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:12 2022] DELETE /identity/v3/users/32f50eb99f1046529f81cc2c91a5f879 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:12.913060 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58e7f3-ea42-4b39-a86a-799facfbe38e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:12.913558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58e7f3-ea42-4b39-a86a-799facfbe38e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:12.913668 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58e7f3-ea42-4b39-a86a-799facfbe38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:12.933660 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4d58e7f3-ea42-4b39-a86a-799facfbe38e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:12.934866 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4d58e7f3-ea42-4b39-a86a-799facfbe38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:12.994135 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1228/2448] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:13.005795 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54490a46-de67-4abe-a67c-144ab46233bb admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:13.007941 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-54490a46-de67-4abe-a67c-144ab46233bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:13.069144 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54490a46-de67-4abe-a67c-144ab46233bb admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:13.072710 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bd661-c623-44f0-9a3e-575393e1fc8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:13.073307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bd661-c623-44f0-9a3e-575393e1fc8b service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:13.073584 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bd661-c623-44f0-9a3e-575393e1fc8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:13.073849 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bd661-c623-44f0-9a3e-575393e1fc8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:13.082738 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bd661-c623-44f0-9a3e-575393e1fc8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:13.084842 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bd661-c623-44f0-9a3e-575393e1fc8b service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:13.094169 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1221/2449] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:13.384882 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58e7f3-ea42-4b39-a86a-799facfbe38e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:13.386807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4d58e7f3-ea42-4b39-a86a-799facfbe38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:13.407171 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58e7f3-ea42-4b39-a86a-799facfbe38e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:13.413290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47930ee-da9c-4bb6-9d05-c747a3ba9440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:13.414015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47930ee-da9c-4bb6-9d05-c747a3ba9440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:13.414230 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47930ee-da9c-4bb6-9d05-c747a3ba9440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:13.414506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47930ee-da9c-4bb6-9d05-c747a3ba9440 admin admin] PATH_INFO: `/v3/projects/9db2ed6b104f4f5887cdeba0da6cf3ec` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:13.415710 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47930ee-da9c-4bb6-9d05-c747a3ba9440 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9db2ed6b104f4f5887cdeba0da6cf3ec)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:13.417713 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47930ee-da9c-4bb6-9d05-c747a3ba9440 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:13.469069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:13.474153 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b47930ee-da9c-4bb6-9d05-c747a3ba9440 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9db2ed6b104f4f5887cdeba0da6cf3ec', 'request_id': 'req-b47930ee-da9c-4bb6-9d05-c747a3ba9440'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.480811 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b47930ee-da9c-4bb6-9d05-c747a3ba9440 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9db2ed6b104f4f5887cdeba0da6cf3ec', 'request_id': 'req-b47930ee-da9c-4bb6-9d05-c747a3ba9440'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.488935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b47930ee-da9c-4bb6-9d05-c747a3ba9440 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9db2ed6b104f4f5887cdeba0da6cf3ec', 'request_id': 'req-b47930ee-da9c-4bb6-9d05-c747a3ba9440'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.500908 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b47930ee-da9c-4bb6-9d05-c747a3ba9440 admin admin] The token cache is being invalidate because project 9db2ed6b104f4f5887cdeba0da6cf3ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:13.501232 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b47930ee-da9c-4bb6-9d05-c747a3ba9440 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.501834 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b47930ee-da9c-4bb6-9d05-c747a3ba9440 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.503378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1229/2450] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:13 2022] DELETE /identity/v3/projects/9db2ed6b104f4f5887cdeba0da6cf3ec => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:13.507625 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bd661-c623-44f0-9a3e-575393e1fc8b service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:13.517900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bd661-c623-44f0-9a3e-575393e1fc8b service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:13.583962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1fc48-3493-4314-8b7c-6291d0bb85b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:13.584754 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1fc48-3493-4314-8b7c-6291d0bb85b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:13.585077 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1fc48-3493-4314-8b7c-6291d0bb85b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:13.585356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1fc48-3493-4314-8b7c-6291d0bb85b4 admin admin] PATH_INFO: `/v3/users/eecee98a53354225b8c7acae5cfd13ac` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:13.586690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1fc48-3493-4314-8b7c-6291d0bb85b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eecee98a53354225b8c7acae5cfd13ac)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:13.588695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1fc48-3493-4314-8b7c-6291d0bb85b4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:13.622402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:13.647252 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-48c1fc48-3493-4314-8b7c-6291d0bb85b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eecee98a53354225b8c7acae5cfd13ac'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.656257 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-48c1fc48-3493-4314-8b7c-6291d0bb85b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eecee98a53354225b8c7acae5cfd13ac'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.670262 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-48c1fc48-3493-4314-8b7c-6291d0bb85b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eecee98a53354225b8c7acae5cfd13ac'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.677859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-48c1fc48-3493-4314-8b7c-6291d0bb85b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eecee98a53354225b8c7acae5cfd13ac'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.678579 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-48c1fc48-3493-4314-8b7c-6291d0bb85b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eecee98a53354225b8c7acae5cfd13ac'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:13.684512 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1222/2451] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:13 2022] DELETE /identity/v3/users/eecee98a53354225b8c7acae5cfd13ac => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:13.987408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47930ee-da9c-4bb6-9d05-c747a3ba9440 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:13.989422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b47930ee-da9c-4bb6-9d05-c747a3ba9440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:14.010027 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47930ee-da9c-4bb6-9d05-c747a3ba9440 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:14.074197 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed1cd0-b02a-48e0-93dc-7d7d6aa997b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:14.074839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed1cd0-b02a-48e0-93dc-7d7d6aa997b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:14.075114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed1cd0-b02a-48e0-93dc-7d7d6aa997b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:14.075383 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed1cd0-b02a-48e0-93dc-7d7d6aa997b2 admin admin] PATH_INFO: `/v3/projects/ee58336bb8164d06a9ccdd29d1ec293e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:14.076550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bed1cd0-b02a-48e0-93dc-7d7d6aa997b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee58336bb8164d06a9ccdd29d1ec293e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:14.078449 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bed1cd0-b02a-48e0-93dc-7d7d6aa997b2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:14.127756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5bed1cd0-b02a-48e0-93dc-7d7d6aa997b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee58336bb8164d06a9ccdd29d1ec293e', 'request_id': 'req-5bed1cd0-b02a-48e0-93dc-7d7d6aa997b2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.134330 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5bed1cd0-b02a-48e0-93dc-7d7d6aa997b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee58336bb8164d06a9ccdd29d1ec293e', 'request_id': 'req-5bed1cd0-b02a-48e0-93dc-7d7d6aa997b2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.142235 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5bed1cd0-b02a-48e0-93dc-7d7d6aa997b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee58336bb8164d06a9ccdd29d1ec293e', 'request_id': 'req-5bed1cd0-b02a-48e0-93dc-7d7d6aa997b2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.153967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5bed1cd0-b02a-48e0-93dc-7d7d6aa997b2 admin admin] The token cache is being invalidate because project ee58336bb8164d06a9ccdd29d1ec293e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:14.154253 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5bed1cd0-b02a-48e0-93dc-7d7d6aa997b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.154896 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5bed1cd0-b02a-48e0-93dc-7d7d6aa997b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.156482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1230/2452] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:13 2022] DELETE /identity/v3/projects/ee58336bb8164d06a9ccdd29d1ec293e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:14.160530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1fc48-3493-4314-8b7c-6291d0bb85b4 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:14.162058 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-48c1fc48-3493-4314-8b7c-6291d0bb85b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:14.174600 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1fc48-3493-4314-8b7c-6291d0bb85b4 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:14.238538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d966ac3-4847-49a1-9e8e-7ac5b8ab2556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:14.239076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d966ac3-4847-49a1-9e8e-7ac5b8ab2556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:14.240084 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d966ac3-4847-49a1-9e8e-7ac5b8ab2556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:14.240437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d966ac3-4847-49a1-9e8e-7ac5b8ab2556 admin admin] PATH_INFO: `/v3/users/119f48ecfb2f4da3a13ff5ad3ff8e141` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:14.241836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d966ac3-4847-49a1-9e8e-7ac5b8ab2556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=119f48ecfb2f4da3a13ff5ad3ff8e141)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:14.243816 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d966ac3-4847-49a1-9e8e-7ac5b8ab2556 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:14.295134 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3d966ac3-4847-49a1-9e8e-7ac5b8ab2556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '119f48ecfb2f4da3a13ff5ad3ff8e141'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.304188 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3d966ac3-4847-49a1-9e8e-7ac5b8ab2556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '119f48ecfb2f4da3a13ff5ad3ff8e141'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.317108 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3d966ac3-4847-49a1-9e8e-7ac5b8ab2556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '119f48ecfb2f4da3a13ff5ad3ff8e141'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.324233 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3d966ac3-4847-49a1-9e8e-7ac5b8ab2556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '119f48ecfb2f4da3a13ff5ad3ff8e141'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.324946 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-3d966ac3-4847-49a1-9e8e-7ac5b8ab2556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '119f48ecfb2f4da3a13ff5ad3ff8e141'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.330963 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1223/2453] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:04:14 2022] DELETE /identity/v3/users/119f48ecfb2f4da3a13ff5ad3ff8e141 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:14.609159 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed1cd0-b02a-48e0-93dc-7d7d6aa997b2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:14.612694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5bed1cd0-b02a-48e0-93dc-7d7d6aa997b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:14.629654 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed1cd0-b02a-48e0-93dc-7d7d6aa997b2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:14.698873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673a088-ec69-416d-81fe-aa43da14b6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:14.699510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673a088-ec69-416d-81fe-aa43da14b6fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:14.699786 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673a088-ec69-416d-81fe-aa43da14b6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:14.700062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673a088-ec69-416d-81fe-aa43da14b6fb admin admin] PATH_INFO: `/v3/projects/8ec243bacf9447bc897ada70d2474d11` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:14.701207 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673a088-ec69-416d-81fe-aa43da14b6fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ec243bacf9447bc897ada70d2474d11)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:14.703101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673a088-ec69-416d-81fe-aa43da14b6fb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:14.705838 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:14.759278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a673a088-ec69-416d-81fe-aa43da14b6fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ec243bacf9447bc897ada70d2474d11', 'request_id': 'req-a673a088-ec69-416d-81fe-aa43da14b6fb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.765167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a673a088-ec69-416d-81fe-aa43da14b6fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ec243bacf9447bc897ada70d2474d11', 'request_id': 'req-a673a088-ec69-416d-81fe-aa43da14b6fb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.773668 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a673a088-ec69-416d-81fe-aa43da14b6fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ec243bacf9447bc897ada70d2474d11', 'request_id': 'req-a673a088-ec69-416d-81fe-aa43da14b6fb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.787436 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a673a088-ec69-416d-81fe-aa43da14b6fb admin admin] The token cache is being invalidate because project 8ec243bacf9447bc897ada70d2474d11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:04:14.787634 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a673a088-ec69-416d-81fe-aa43da14b6fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.788298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a673a088-ec69-416d-81fe-aa43da14b6fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:04:14.789822 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1231/2454] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:04:14 2022] DELETE /identity/v3/projects/8ec243bacf9447bc897ada70d2474d11 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:04:14.794914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5358d5a-b23d-4041-9e19-e0780e56a05e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:14.795206 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5358d5a-b23d-4041-9e19-e0780e56a05e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:14.795507 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5358d5a-b23d-4041-9e19-e0780e56a05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:14.815740 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b5358d5a-b23d-4041-9e19-e0780e56a05e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:14.818188 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b5358d5a-b23d-4041-9e19-e0780e56a05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:14.836415 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:14.885134 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1224/2455] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:14.893322 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c410e9-f651-47e8-a368-6b03ab330db1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:14.893607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c410e9-f651-47e8-a368-6b03ab330db1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:14.893883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c410e9-f651-47e8-a368-6b03ab330db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:14.914321 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-69c410e9-f651-47e8-a368-6b03ab330db1 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:14.915575 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-69c410e9-f651-47e8-a368-6b03ab330db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:14.923714 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1232/2456] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:04:14.928880 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5358d5a-b23d-4041-9e19-e0780e56a05e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:14.938424 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5358d5a-b23d-4041-9e19-e0780e56a05e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:14.943939 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb788fa-1e2f-44b6-aef2-a0628c75c25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:14.944561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb788fa-1e2f-44b6-aef2-a0628c75c25b admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:14.944836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb788fa-1e2f-44b6-aef2-a0628c75c25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:14.945106 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb788fa-1e2f-44b6-aef2-a0628c75c25b admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:14.945738 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb788fa-1e2f-44b6-aef2-a0628c75c25b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:04:14.946040 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb788fa-1e2f-44b6-aef2-a0628c75c25b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:14.947893 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb788fa-1e2f-44b6-aef2-a0628c75c25b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:14.959744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1225/2457] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:04:14.964074 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c410e9-f651-47e8-a368-6b03ab330db1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:14.973359 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c410e9-f651-47e8-a368-6b03ab330db1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:14.978839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2890d69-579b-4389-aaf8-542874c8c23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:14.979435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2890d69-579b-4389-aaf8-542874c8c23c admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:14.979718 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2890d69-579b-4389-aaf8-542874c8c23c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:14.980755 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2890d69-579b-4389-aaf8-542874c8c23c admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:14.981481 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2890d69-579b-4389-aaf8-542874c8c23c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:14.983441 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2890d69-579b-4389-aaf8-542874c8c23c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:15.016377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1233/2458] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:04:14 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:15.017679 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb788fa-1e2f-44b6-aef2-a0628c75c25b admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:15.028638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb788fa-1e2f-44b6-aef2-a0628c75c25b admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:15.093829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b05fc-fcb0-4e30-a573-3c6276cda8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:15.094453 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b05fc-fcb0-4e30-a573-3c6276cda8bd admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:15.094734 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b05fc-fcb0-4e30-a573-3c6276cda8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:15.095002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b05fc-fcb0-4e30-a573-3c6276cda8bd admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:15.095637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b05fc-fcb0-4e30-a573-3c6276cda8bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:15.097574 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b05fc-fcb0-4e30-a573-3c6276cda8bd admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:15.128142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1226/2459] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:04:15 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:04:15.128676 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2890d69-579b-4389-aaf8-542874c8c23c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:15.142663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2890d69-579b-4389-aaf8-542874c8c23c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:15.148615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c95ebe-db57-4b75-ac60-3c37d91aefcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:15.149478 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c95ebe-db57-4b75-ac60-3c37d91aefcb admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:15.149822 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c95ebe-db57-4b75-ac60-3c37d91aefcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:15.150157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c95ebe-db57-4b75-ac60-3c37d91aefcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:15.150875 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c95ebe-db57-4b75-ac60-3c37d91aefcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:15.153277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c95ebe-db57-4b75-ac60-3c37d91aefcb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:15.164354 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1234/2460] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:04:15.169018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b05fc-fcb0-4e30-a573-3c6276cda8bd admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:15.179156 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b05fc-fcb0-4e30-a573-3c6276cda8bd admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:15.184121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42f49a-dbd0-42ac-9f36-c40bf9949991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:15.184752 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42f49a-dbd0-42ac-9f36-c40bf9949991 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:15.185031 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42f49a-dbd0-42ac-9f36-c40bf9949991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:15.185305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42f49a-dbd0-42ac-9f36-c40bf9949991 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:15.185991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa42f49a-dbd0-42ac-9f36-c40bf9949991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:15.187669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa42f49a-dbd0-42ac-9f36-c40bf9949991 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:15.199583 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1227/2461] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:04:15.203628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c95ebe-db57-4b75-ac60-3c37d91aefcb admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:15.214323 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c95ebe-db57-4b75-ac60-3c37d91aefcb admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:15.219153 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e77aaa-4c60-447d-bc7e-269e0e87980b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:15.219800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e77aaa-4c60-447d-bc7e-269e0e87980b admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:15.220079 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e77aaa-4c60-447d-bc7e-269e0e87980b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:15.220385 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e77aaa-4c60-447d-bc7e-269e0e87980b admin admin] PATH_INFO: `/v3/projects/3e4a52a4c3744faca37f92db4493357a/users/a45d5f4ba3284323bb58bb90625ae94f/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:15.235728 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e77aaa-4c60-447d-bc7e-269e0e87980b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e4a52a4c3744faca37f92db4493357a, user_id=a45d5f4ba3284323bb58bb90625ae94f, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:15.238427 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e77aaa-4c60-447d-bc7e-269e0e87980b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:15.251100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1235/2462] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:04:15 2022] PUT /identity/v3/projects/3e4a52a4c3744faca37f92db4493357a/users/a45d5f4ba3284323bb58bb90625ae94f/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:04:15.257195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ee535-e71a-4b86-813f-7d661eb5f252 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:15.257514 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ee535-e71a-4b86-813f-7d661eb5f252 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:15.257788 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ee535-e71a-4b86-813f-7d661eb5f252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:15.277357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-269ee535-e71a-4b86-813f-7d661eb5f252 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:15.278884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-269ee535-e71a-4b86-813f-7d661eb5f252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:15.337078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1228/2463] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:15.346759 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e77aaa-4c60-447d-bc7e-269e0e87980b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:15.348484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e0e77aaa-4c60-447d-bc7e-269e0e87980b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:15.406149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e77aaa-4c60-447d-bc7e-269e0e87980b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:15.409802 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135a8f9-fae8-4972-8fdc-7d5f1a2775d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:15.410411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135a8f9-fae8-4972-8fdc-7d5f1a2775d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:15.410692 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135a8f9-fae8-4972-8fdc-7d5f1a2775d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:15.410963 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135a8f9-fae8-4972-8fdc-7d5f1a2775d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:15.420417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7135a8f9-fae8-4972-8fdc-7d5f1a2775d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:15.421838 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7135a8f9-fae8-4972-8fdc-7d5f1a2775d4 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:15.431917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1236/2464] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:04:16.105089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9fbfb-9ec1-4aec-be57-3e0776d6233c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:16.105897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9fbfb-9ec1-4aec-be57-3e0776d6233c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:16.106516 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9fbfb-9ec1-4aec-be57-3e0776d6233c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:16.147099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-00a9fbfb-9ec1-4aec-be57-3e0776d6233c None None] MFA Rules not processed for user `a45d5f4ba3284323bb58bb90625ae94f`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:16.148459 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-00a9fbfb-9ec1-4aec-be57-3e0776d6233c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:16.166980 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:16.221481 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1229/2465] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:04:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:16.233849 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135a8f9-fae8-4972-8fdc-7d5f1a2775d4 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:04:16.235356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7135a8f9-fae8-4972-8fdc-7d5f1a2775d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:16.275587 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:16.304961 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135a8f9-fae8-4972-8fdc-7d5f1a2775d4 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:04:16.308884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ff9d9-f0d9-4e4e-952e-7bf2d8650645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:04:16.309510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ff9d9-f0d9-4e4e-952e-7bf2d8650645 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:16.309873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ff9d9-f0d9-4e4e-952e-7bf2d8650645 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:16.310055 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ff9d9-f0d9-4e4e-952e-7bf2d8650645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:16.319570 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ff9d9-f0d9-4e4e-952e-7bf2d8650645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:04:16.321061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ff9d9-f0d9-4e4e-952e-7bf2d8650645 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:04:16.331122 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1237/2466] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:04:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:04:17.703651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3775b-d56b-4ef0-b5dd-8e39cc9d7f8f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:17.704216 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3775b-d56b-4ef0-b5dd-8e39cc9d7f8f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:17.704216 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3775b-d56b-4ef0-b5dd-8e39cc9d7f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:17.724604 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d3d3775b-d56b-4ef0-b5dd-8e39cc9d7f8f None None] MFA Rules not processed for user `a45d5f4ba3284323bb58bb90625ae94f`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:17.725805 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d3d3775b-d56b-4ef0-b5dd-8e39cc9d7f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:17.733172 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1230/2467] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:04:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:04:26.327301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=12 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:27.721073 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=22 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:04:50.303025 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16560db7-b6e0-4f9d-b0a1-03c6e68c3380 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:04:50.303875 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16560db7-b6e0-4f9d-b0a1-03c6e68c3380 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:04:50.303875 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16560db7-b6e0-4f9d-b0a1-03c6e68c3380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:04:50.328076 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-16560db7-b6e0-4f9d-b0a1-03c6e68c3380 None None] MFA Rules not processed for user `bd0929f371e4442eb4eb6e12a96bbfc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:04:50.329265 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-16560db7-b6e0-4f9d-b0a1-03c6e68c3380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:04:50.392101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1238/2468] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:04:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:00.383914 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=25 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:03.396948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3775b-d56b-4ef0-b5dd-8e39cc9d7f8f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:03.398961 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d3d3775b-d56b-4ef0-b5dd-8e39cc9d7f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:03.514694 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3775b-d56b-4ef0-b5dd-8e39cc9d7f8f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:03.522101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6993d5-2242-4e4a-97e9-8e1960ae2d6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:03.522822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6993d5-2242-4e4a-97e9-8e1960ae2d6b service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:03.523106 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6993d5-2242-4e4a-97e9-8e1960ae2d6b service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:03.523378 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6993d5-2242-4e4a-97e9-8e1960ae2d6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:03.525169 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-da6993d5-2242-4e4a-97e9-8e1960ae2d6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:03.538584 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6993d5-2242-4e4a-97e9-8e1960ae2d6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:03.540381 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6993d5-2242-4e4a-97e9-8e1960ae2d6b service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:03.556403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1231/2469] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:13.549098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=46 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:17.304600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceebceb-b06c-4307-8a34-865fb09b4b27 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:17.305159 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceebceb-b06c-4307-8a34-865fb09b4b27 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:17.305405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceebceb-b06c-4307-8a34-865fb09b4b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:17.333959 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5ceebceb-b06c-4307-8a34-865fb09b4b27 None None] MFA Rules not processed for user `b11abed8d4424a1e9957c3dc9aefadd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:17.335151 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5ceebceb-b06c-4307-8a34-865fb09b4b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:17.399853 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1239/2470] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 24 09:05:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:17.446072 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6993d5-2242-4e4a-97e9-8e1960ae2d6b service nova] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:17.447503 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-da6993d5-2242-4e4a-97e9-8e1960ae2d6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:17.519989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6993d5-2242-4e4a-97e9-8e1960ae2d6b service nova] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:17.525606 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5070ac5-0855-45aa-b5d7-513becb56e87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46b1417d4ac24c8982b2052f4e99042a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:17.526227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5070ac5-0855-45aa-b5d7-513becb56e87 service swift] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:17.526509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5070ac5-0855-45aa-b5d7-513becb56e87 service swift] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:17.526801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5070ac5-0855-45aa-b5d7-513becb56e87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:17.536575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5070ac5-0855-45aa-b5d7-513becb56e87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:17.538257 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5070ac5-0855-45aa-b5d7-513becb56e87 service swift] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:17.546977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1232/2471] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 24 09:05:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:05:26.621211 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceebceb-b06c-4307-8a34-865fb09b4b27 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:26.631350 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceebceb-b06c-4307-8a34-865fb09b4b27 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:26.635139 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ff92e-2f57-4e9f-a81e-6808df3261ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:26.635816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ff92e-2f57-4e9f-a81e-6808df3261ba service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:26.636131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ff92e-2f57-4e9f-a81e-6808df3261ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:26.636404 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ff92e-2f57-4e9f-a81e-6808df3261ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:26.638431 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9e7ff92e-2f57-4e9f-a81e-6808df3261ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:26.708764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ff92e-2f57-4e9f-a81e-6808df3261ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:26.710290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ff92e-2f57-4e9f-a81e-6808df3261ba service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:26.726459 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1240/2472] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:27.543103 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=32 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:30.977217 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5070ac5-0855-45aa-b5d7-513becb56e87 service swift] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:30.978656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d5070ac5-0855-45aa-b5d7-513becb56e87 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:30.990759 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5070ac5-0855-45aa-b5d7-513becb56e87 service swift] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:30.994330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7249f-366e-4f77-83a5-3167fdef7b20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:30.994880 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7249f-366e-4f77-83a5-3167fdef7b20 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:30.995156 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7249f-366e-4f77-83a5-3167fdef7b20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:30.995421 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7249f-366e-4f77-83a5-3167fdef7b20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:30.997137 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-75a7249f-366e-4f77-83a5-3167fdef7b20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:31.010268 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7249f-366e-4f77-83a5-3167fdef7b20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:31.011676 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7249f-366e-4f77-83a5-3167fdef7b20 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:31.022927 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1233/2473] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:33.401737 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ff92e-2f57-4e9f-a81e-6808df3261ba service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:33.436328 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ff92e-2f57-4e9f-a81e-6808df3261ba service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:33.442071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ebdaa-911f-4be3-a46b-d8e6df2a9f5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:33.442933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ebdaa-911f-4be3-a46b-d8e6df2a9f5f service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:33.443420 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ebdaa-911f-4be3-a46b-d8e6df2a9f5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:33.443673 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ebdaa-911f-4be3-a46b-d8e6df2a9f5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:33.445515 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-795ebdaa-911f-4be3-a46b-d8e6df2a9f5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:33.477788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ebdaa-911f-4be3-a46b-d8e6df2a9f5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:33.479657 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ebdaa-911f-4be3-a46b-d8e6df2a9f5f service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:33.494418 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1241/2474] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:38.157928 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3081a6-9092-4b43-865b-6e49ad3c12fd None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:38.158478 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3081a6-9092-4b43-865b-6e49ad3c12fd None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:38.158478 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3081a6-9092-4b43-865b-6e49ad3c12fd None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:38.159957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1234/2475] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 24 09:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:05:38.165896 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59971c9d-8124-4b62-b307-06d1b94d4af1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:38.166385 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59971c9d-8124-4b62-b307-06d1b94d4af1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:38.166488 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59971c9d-8124-4b62-b307-06d1b94d4af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:38.206793 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-59971c9d-8124-4b62-b307-06d1b94d4af1 None None] MFA Rules not processed for user `ca56dd7216fb4946946b2801e144fec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:38.207927 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-59971c9d-8124-4b62-b307-06d1b94d4af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:38.217968 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1242/2476] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 24 09:05:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:05:38.229909 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3081a6-9092-4b43-865b-6e49ad3c12fd None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:38.231658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-5e3081a6-9092-4b43-865b-6e49ad3c12fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:38.293429 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3081a6-9092-4b43-865b-6e49ad3c12fd None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:38.297925 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271f7be-8115-49a4-bbb6-9be4dd4f95a6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5979387f7aa54619b8d429368c306cc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:38.298517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271f7be-8115-49a4-bbb6-9be4dd4f95a6 service glance] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:38.298804 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271f7be-8115-49a4-bbb6-9be4dd4f95a6 service glance] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:38.299097 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271f7be-8115-49a4-bbb6-9be4dd4f95a6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:38.309605 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6271f7be-8115-49a4-bbb6-9be4dd4f95a6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:38.311049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6271f7be-8115-49a4-bbb6-9be4dd4f95a6 service glance] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:38.321711 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1235/2477] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:38.391567 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee73c6-2b26-484f-a5a8-03a88bbe2023 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:38.391853 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee73c6-2b26-484f-a5a8-03a88bbe2023 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:38.392164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee73c6-2b26-484f-a5a8-03a88bbe2023 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:38.393525 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1243/2478] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 24 09:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:05:38.398337 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848e5d0-4f8e-472e-a90a-e23065fb7438 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:38.398628 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848e5d0-4f8e-472e-a90a-e23065fb7438 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:38.398877 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848e5d0-4f8e-472e-a90a-e23065fb7438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:38.419428 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-7848e5d0-4f8e-472e-a90a-e23065fb7438 None None] MFA Rules not processed for user `2ee2e93901914b1d82607a3ae0e2acf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:38.420860 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7848e5d0-4f8e-472e-a90a-e23065fb7438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:38.427434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1236/2479] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 24 09:05:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:38.439253 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee73c6-2b26-484f-a5a8-03a88bbe2023 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:38.450319 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee73c6-2b26-484f-a5a8-03a88bbe2023 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:38.454572 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e3c1c-cd68-4333-a3bf-e2bb352b68ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:38.455169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e3c1c-cd68-4333-a3bf-e2bb352b68ea service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:38.455448 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e3c1c-cd68-4333-a3bf-e2bb352b68ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:38.455722 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e3c1c-cd68-4333-a3bf-e2bb352b68ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:38.463556 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e3c1c-cd68-4333-a3bf-e2bb352b68ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:38.464940 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e3c1c-cd68-4333-a3bf-e2bb352b68ea service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:38.473910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1244/2480] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:38.559465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf40b88-3199-438f-8162-dbb3eba39f8f None None] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:38.559753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf40b88-3199-438f-8162-dbb3eba39f8f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:38.560041 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf40b88-3199-438f-8162-dbb3eba39f8f None None] PATH_INFO: `/` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:38.561582 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1237/2481] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 24 09:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:05:38.567125 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c12918-3144-4c68-a8e4-ad3864b19f46 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:38.567481 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c12918-3144-4c68-a8e4-ad3864b19f46 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:38.567821 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c12918-3144-4c68-a8e4-ad3864b19f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:38.588385 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d0c12918-3144-4c68-a8e4-ad3864b19f46 None None] MFA Rules not processed for user `ca56dd7216fb4946946b2801e144fec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:38.589589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d0c12918-3144-4c68-a8e4-ad3864b19f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:38.599379 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1245/2482] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 24 09:05:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:05:38.610107 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf40b88-3199-438f-8162-dbb3eba39f8f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:38.621177 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf40b88-3199-438f-8162-dbb3eba39f8f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:38.625144 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93e049-e494-4fff-87c4-9d73f2531d0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:38.625755 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93e049-e494-4fff-87c4-9d73f2531d0f service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:38.626034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93e049-e494-4fff-87c4-9d73f2531d0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:38.626303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93e049-e494-4fff-87c4-9d73f2531d0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:38.636889 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93e049-e494-4fff-87c4-9d73f2531d0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:38.638868 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93e049-e494-4fff-87c4-9d73f2531d0f service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:38.650071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1238/2483] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:40.152927 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde18cf6-8b95-4863-8857-91aa06726ea9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:40.154048 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde18cf6-8b95-4863-8857-91aa06726ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:40.154351 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde18cf6-8b95-4863-8857-91aa06726ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:40.174307 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-cde18cf6-8b95-4863-8857-91aa06726ea9 None None] MFA Rules not processed for user `46b3aa99a5e84ef2a8df193628feb0d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:40.175648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cde18cf6-8b95-4863-8857-91aa06726ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:40.189251 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1246/2484] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:05:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:40.593312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80a782-bb52-4393-a91d-d5adb48dcd95 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:40.593862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80a782-bb52-4393-a91d-d5adb48dcd95 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:40.593862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80a782-bb52-4393-a91d-d5adb48dcd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:40.616971 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-dd80a782-bb52-4393-a91d-d5adb48dcd95 None None] MFA Rules not processed for user `b11abed8d4424a1e9957c3dc9aefadd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:05:40.618312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dd80a782-bb52-4393-a91d-d5adb48dcd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:40.626910 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1239/2485] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 24 09:05:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:05:40.639798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde18cf6-8b95-4863-8857-91aa06726ea9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:40.646364 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=85 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:40.653890 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde18cf6-8b95-4863-8857-91aa06726ea9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:40.659525 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4856b0-105a-461e-8223-4df8e6588800 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46b1417d4ac24c8982b2052f4e99042a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:40.660258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4856b0-105a-461e-8223-4df8e6588800 service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:40.660608 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4856b0-105a-461e-8223-4df8e6588800 service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:40.660995 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4856b0-105a-461e-8223-4df8e6588800 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:40.670551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4856b0-105a-461e-8223-4df8e6588800 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:40.672309 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4856b0-105a-461e-8223-4df8e6588800 service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:40.680753 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1247/2486] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 24 09:05:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:05:46.565335 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80a782-bb52-4393-a91d-d5adb48dcd95 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:05:46.566849 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dd80a782-bb52-4393-a91d-d5adb48dcd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:46.582729 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80a782-bb52-4393-a91d-d5adb48dcd95 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:05:46.588349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eecbfa-645a-4a8a-a5df-34f430c4957d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:05:46.589140 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eecbfa-645a-4a8a-a5df-34f430c4957d service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:05:46.589479 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eecbfa-645a-4a8a-a5df-34f430c4957d service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:05:46.589656 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eecbfa-645a-4a8a-a5df-34f430c4957d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:05:46.591616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-82eecbfa-645a-4a8a-a5df-34f430c4957d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:05:46.691784 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eecbfa-645a-4a8a-a5df-34f430c4957d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:05:46.693890 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eecbfa-645a-4a8a-a5df-34f430c4957d service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:05:46.729349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1240/2487] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:05:46 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:05:50.677460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:05:56.712736 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=85 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:04.312064 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296986e-ef65-4915-bcfa-2690c553990d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:04.312644 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296986e-ef65-4915-bcfa-2690c553990d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:04.312644 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296986e-ef65-4915-bcfa-2690c553990d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:04.362622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b296986e-ef65-4915-bcfa-2690c553990d None None] MFA Rules not processed for user `b11abed8d4424a1e9957c3dc9aefadd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:06:04.364327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b296986e-ef65-4915-bcfa-2690c553990d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:04.382190 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1248/2488] 10.222.0.67 () {58 vars in 967 bytes} [Wed Aug 24 09:06:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:06:04.395444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eecbfa-645a-4a8a-a5df-34f430c4957d service nova] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:04.417908 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eecbfa-645a-4a8a-a5df-34f430c4957d service nova] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:04.422994 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2519cf-d480-4f7a-a14a-f4174688de50 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46b1417d4ac24c8982b2052f4e99042a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:04.423604 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2519cf-d480-4f7a-a14a-f4174688de50 service swift] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:04.423882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2519cf-d480-4f7a-a14a-f4174688de50 service swift] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:04.424187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2519cf-d480-4f7a-a14a-f4174688de50 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:04.433493 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2519cf-d480-4f7a-a14a-f4174688de50 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:04.435256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2519cf-d480-4f7a-a14a-f4174688de50 service swift] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:04.446733 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1241/2489] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 24 09:06:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:06:08.940672 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296986e-ef65-4915-bcfa-2690c553990d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:08.950436 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296986e-ef65-4915-bcfa-2690c553990d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:08.954570 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7cab7-54dd-40e2-97e9-6ce4acfa5e6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:08.955730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7cab7-54dd-40e2-97e9-6ce4acfa5e6d service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:08.956011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7cab7-54dd-40e2-97e9-6ce4acfa5e6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:08.956318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7cab7-54dd-40e2-97e9-6ce4acfa5e6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:08.958719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-25d7cab7-54dd-40e2-97e9-6ce4acfa5e6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:08.969063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d7cab7-54dd-40e2-97e9-6ce4acfa5e6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:08.971127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d7cab7-54dd-40e2-97e9-6ce4acfa5e6d service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:08.979909 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1249/2490] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:14.442204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=10 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:17.846230 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e650d-5205-41f7-90e5-30ed5d327aff None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:17.846832 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e650d-5205-41f7-90e5-30ed5d327aff None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:17.846832 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e650d-5205-41f7-90e5-30ed5d327aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:17.865749 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-544e650d-5205-41f7-90e5-30ed5d327aff None None] MFA Rules not processed for user `a45d5f4ba3284323bb58bb90625ae94f`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:06:17.866940 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-544e650d-5205-41f7-90e5-30ed5d327aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:17.873882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1242/2491] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:06:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:06:18.139501 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7cab7-54dd-40e2-97e9-6ce4acfa5e6d service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:18.148077 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7cab7-54dd-40e2-97e9-6ce4acfa5e6d service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:18.151785 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af668e-ad9c-48fe-be2e-334dc1ca8f10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:18.152521 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af668e-ad9c-48fe-be2e-334dc1ca8f10 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:18.152871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af668e-ad9c-48fe-be2e-334dc1ca8f10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:18.153210 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af668e-ad9c-48fe-be2e-334dc1ca8f10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:18.155099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b4af668e-ad9c-48fe-be2e-334dc1ca8f10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:18.172377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af668e-ad9c-48fe-be2e-334dc1ca8f10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:18.173845 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af668e-ad9c-48fe-be2e-334dc1ca8f10 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:18.182315 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1250/2492] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:27.860485 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:28.178574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=30 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:29.237429 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e650d-5205-41f7-90e5-30ed5d327aff None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:29.251686 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e650d-5205-41f7-90e5-30ed5d327aff None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:29.255879 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7a907-fede-43a8-a687-90a0312116e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:29.256540 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7a907-fede-43a8-a687-90a0312116e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:29.256811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7a907-fede-43a8-a687-90a0312116e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:29.257076 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7a907-fede-43a8-a687-90a0312116e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:29.260087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-93d7a907-fede-43a8-a687-90a0312116e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:29.330763 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d7a907-fede-43a8-a687-90a0312116e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:29.332232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d7a907-fede-43a8-a687-90a0312116e6 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:29.343808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1243/2493] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:29 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:36.068807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af668e-ad9c-48fe-be2e-334dc1ca8f10 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:36.083697 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af668e-ad9c-48fe-be2e-334dc1ca8f10 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:36.093184 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468300a-c439-4a71-a459-c42ab5323445 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:36.093818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468300a-c439-4a71-a459-c42ab5323445 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:36.094103 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468300a-c439-4a71-a459-c42ab5323445 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:36.094405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468300a-c439-4a71-a459-c42ab5323445 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:36.096230 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0468300a-c439-4a71-a459-c42ab5323445 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:36.109818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468300a-c439-4a71-a459-c42ab5323445 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:36.111986 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468300a-c439-4a71-a459-c42ab5323445 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:36.128766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1251/2494] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:36.409135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7a907-fede-43a8-a687-90a0312116e6 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:36.422212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7a907-fede-43a8-a687-90a0312116e6 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:36.430105 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2251d51c-5ea2-4df1-a10c-c1bc9c602dca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:36.430755 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251d51c-5ea2-4df1-a10c-c1bc9c602dca service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:36.431018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251d51c-5ea2-4df1-a10c-c1bc9c602dca service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:36.431286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251d51c-5ea2-4df1-a10c-c1bc9c602dca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:36.433214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2251d51c-5ea2-4df1-a10c-c1bc9c602dca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:36.528989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2251d51c-5ea2-4df1-a10c-c1bc9c602dca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:36.531384 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2251d51c-5ea2-4df1-a10c-c1bc9c602dca service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:36.572543 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1244/2495] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:41.991666 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468300a-c439-4a71-a459-c42ab5323445 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:42.001852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468300a-c439-4a71-a459-c42ab5323445 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:42.005720 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e5ece-5179-466a-9ad2-bfc471a79fc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:42.006326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e5ece-5179-466a-9ad2-bfc471a79fc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:42.006599 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e5ece-5179-466a-9ad2-bfc471a79fc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:42.007493 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e5ece-5179-466a-9ad2-bfc471a79fc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:42.009367 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-214e5ece-5179-466a-9ad2-bfc471a79fc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:42.021508 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e5ece-5179-466a-9ad2-bfc471a79fc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:42.023511 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e5ece-5179-466a-9ad2-bfc471a79fc8 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:42.033242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1252/2496] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:45.267988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2251d51c-5ea2-4df1-a10c-c1bc9c602dca service nova] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:45.270347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2251d51c-5ea2-4df1-a10c-c1bc9c602dca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:45.331188 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e5ece-5179-466a-9ad2-bfc471a79fc8 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:45.331188 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-214e5ece-5179-466a-9ad2-bfc471a79fc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:45.345189 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e5ece-5179-466a-9ad2-bfc471a79fc8 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:45.350117 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19082a7e-bbfd-4d88-871e-67b2ffc92604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:45.350858 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19082a7e-bbfd-4d88-871e-67b2ffc92604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:45.351141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19082a7e-bbfd-4d88-871e-67b2ffc92604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:45.351415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19082a7e-bbfd-4d88-871e-67b2ffc92604 admin admin] PATH_INFO: `/v3/users/f0dc2dcc5327489c9ecc28a43ab7c0e2` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:45.352610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19082a7e-bbfd-4d88-871e-67b2ffc92604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0dc2dcc5327489c9ecc28a43ab7c0e2)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:45.354567 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19082a7e-bbfd-4d88-871e-67b2ffc92604 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:45.360888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2251d51c-5ea2-4df1-a10c-c1bc9c602dca service nova] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:45.366635 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769d4bc-c7e7-4922-b3c5-7510cf26afbc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33a0844a1c8f4ea4932e12d53e5ee9ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:45.367244 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769d4bc-c7e7-4922-b3c5-7510cf26afbc service placement] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:45.367522 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769d4bc-c7e7-4922-b3c5-7510cf26afbc service placement] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:45.367791 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769d4bc-c7e7-4922-b3c5-7510cf26afbc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:45.369586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6769d4bc-c7e7-4922-b3c5-7510cf26afbc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:45.377949 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:45.383942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769d4bc-c7e7-4922-b3c5-7510cf26afbc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:45.386036 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769d4bc-c7e7-4922-b3c5-7510cf26afbc service placement] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:45.397209 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1245/2497] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:06:45.420512 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-19082a7e-bbfd-4d88-871e-67b2ffc92604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0dc2dcc5327489c9ecc28a43ab7c0e2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:45.421270 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-19082a7e-bbfd-4d88-871e-67b2ffc92604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0dc2dcc5327489c9ecc28a43ab7c0e2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:45.430964 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-19082a7e-bbfd-4d88-871e-67b2ffc92604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0dc2dcc5327489c9ecc28a43ab7c0e2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:45.438778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-19082a7e-bbfd-4d88-871e-67b2ffc92604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0dc2dcc5327489c9ecc28a43ab7c0e2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:45.451392 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-19082a7e-bbfd-4d88-871e-67b2ffc92604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0dc2dcc5327489c9ecc28a43ab7c0e2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:45.457409 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1253/2498] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:06:45 2022] DELETE /identity/v3/users/f0dc2dcc5327489c9ecc28a43ab7c0e2 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:06:45.637651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769d4bc-c7e7-4922-b3c5-7510cf26afbc service placement] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:45.650882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769d4bc-c7e7-4922-b3c5-7510cf26afbc service placement] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:45.710505 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d965c8-6661-43a9-8f6e-c6c6d25ce148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:45.711130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d965c8-6661-43a9-8f6e-c6c6d25ce148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:45.711407 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d965c8-6661-43a9-8f6e-c6c6d25ce148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:45.711681 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d965c8-6661-43a9-8f6e-c6c6d25ce148 admin admin] PATH_INFO: `/v3/projects/bd6e1733fd9d4f5096c35e99dd56945b` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:45.713033 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d965c8-6661-43a9-8f6e-c6c6d25ce148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd6e1733fd9d4f5096c35e99dd56945b)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:45.715449 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d965c8-6661-43a9-8f6e-c6c6d25ce148 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:45.765162 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-67d965c8-6661-43a9-8f6e-c6c6d25ce148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd6e1733fd9d4f5096c35e99dd56945b', 'request_id': 'req-67d965c8-6661-43a9-8f6e-c6c6d25ce148'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:45.773420 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-67d965c8-6661-43a9-8f6e-c6c6d25ce148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd6e1733fd9d4f5096c35e99dd56945b', 'request_id': 'req-67d965c8-6661-43a9-8f6e-c6c6d25ce148'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:45.779720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-67d965c8-6661-43a9-8f6e-c6c6d25ce148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd6e1733fd9d4f5096c35e99dd56945b', 'request_id': 'req-67d965c8-6661-43a9-8f6e-c6c6d25ce148'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:45.792765 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-67d965c8-6661-43a9-8f6e-c6c6d25ce148 admin admin] The token cache is being invalidate because project bd6e1733fd9d4f5096c35e99dd56945b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:06:45.793049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-67d965c8-6661-43a9-8f6e-c6c6d25ce148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:45.793702 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-67d965c8-6661-43a9-8f6e-c6c6d25ce148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:06:45.795210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1246/2499] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:06:45 2022] DELETE /identity/v3/projects/bd6e1733fd9d4f5096c35e99dd56945b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:06:45.800147 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a032a-2528-4e4b-af95-73b9ded9467e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:45.800434 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a032a-2528-4e4b-af95-73b9ded9467e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:45.800707 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a032a-2528-4e4b-af95-73b9ded9467e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:45.821568 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c53a032a-2528-4e4b-af95-73b9ded9467e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:06:45.822783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c53a032a-2528-4e4b-af95-73b9ded9467e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:45.882596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1254/2500] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:06:45.890431 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31125524-44d5-48ba-9731-393978df7fad None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:45.890721 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31125524-44d5-48ba-9731-393978df7fad None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:45.890998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31125524-44d5-48ba-9731-393978df7fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:45.913268 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-31125524-44d5-48ba-9731-393978df7fad None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:06:45.914551 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-31125524-44d5-48ba-9731-393978df7fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:45.922604 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1247/2501] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:06:45.927004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a032a-2528-4e4b-af95-73b9ded9467e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:45.937158 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a032a-2528-4e4b-af95-73b9ded9467e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:45.941730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc893fe-5a69-4581-aeb3-a74a83a56e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:45.942331 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc893fe-5a69-4581-aeb3-a74a83a56e61 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:45.942614 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc893fe-5a69-4581-aeb3-a74a83a56e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:45.942941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc893fe-5a69-4581-aeb3-a74a83a56e61 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:45.943626 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc893fe-5a69-4581-aeb3-a74a83a56e61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:06:45.943934 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc893fe-5a69-4581-aeb3-a74a83a56e61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:45.945552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc893fe-5a69-4581-aeb3-a74a83a56e61 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:45.957124 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1255/2502] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:06:45.962505 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31125524-44d5-48ba-9731-393978df7fad None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:45.971766 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31125524-44d5-48ba-9731-393978df7fad None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:45.977275 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecbffd-0c53-4257-81fd-f83a055cbdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:45.977900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecbffd-0c53-4257-81fd-f83a055cbdad admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:45.978197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecbffd-0c53-4257-81fd-f83a055cbdad admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:45.978468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecbffd-0c53-4257-81fd-f83a055cbdad admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:45.979093 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ecbffd-0c53-4257-81fd-f83a055cbdad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:45.981039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ecbffd-0c53-4257-81fd-f83a055cbdad admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:46.026996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1248/2503] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:06:45 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:06:46.031471 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc893fe-5a69-4581-aeb3-a74a83a56e61 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:46.042627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc893fe-5a69-4581-aeb3-a74a83a56e61 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:46.071886 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:46.102815 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92e2d5-c042-4a4a-b916-682817be8f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:46.103452 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92e2d5-c042-4a4a-b916-682817be8f95 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:46.103712 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92e2d5-c042-4a4a-b916-682817be8f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:46.103986 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92e2d5-c042-4a4a-b916-682817be8f95 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:46.104681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92e2d5-c042-4a4a-b916-682817be8f95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:46.106654 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92e2d5-c042-4a4a-b916-682817be8f95 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:46.134069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1256/2504] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:06:46 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:06:46.136337 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecbffd-0c53-4257-81fd-f83a055cbdad admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:46.150377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecbffd-0c53-4257-81fd-f83a055cbdad admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:46.155719 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac1faf-4a8e-4180-a10c-28fe0e723892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:46.156075 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac1faf-4a8e-4180-a10c-28fe0e723892 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:46.156075 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac1faf-4a8e-4180-a10c-28fe0e723892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:46.156374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac1faf-4a8e-4180-a10c-28fe0e723892 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:46.157008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac1faf-4a8e-4180-a10c-28fe0e723892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:46.158609 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac1faf-4a8e-4180-a10c-28fe0e723892 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:46.170962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1249/2505] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:06:46.176300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92e2d5-c042-4a4a-b916-682817be8f95 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:46.188803 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92e2d5-c042-4a4a-b916-682817be8f95 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:46.194118 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e6664-3b0a-4943-9d73-31d2c365bf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:46.194723 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e6664-3b0a-4943-9d73-31d2c365bf9a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:46.195010 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e6664-3b0a-4943-9d73-31d2c365bf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:46.195282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e6664-3b0a-4943-9d73-31d2c365bf9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:46.195929 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e6664-3b0a-4943-9d73-31d2c365bf9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:46.197545 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e6664-3b0a-4943-9d73-31d2c365bf9a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:46.209657 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1257/2506] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:06:46.213873 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac1faf-4a8e-4180-a10c-28fe0e723892 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:46.225050 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac1faf-4a8e-4180-a10c-28fe0e723892 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:46.229816 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41047565-3e56-4546-9700-82703005cdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:46.230473 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41047565-3e56-4546-9700-82703005cdde admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:46.230757 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41047565-3e56-4546-9700-82703005cdde admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:46.231031 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41047565-3e56-4546-9700-82703005cdde admin admin] PATH_INFO: `/v3/projects/b5ac14f3b99d4f44874e947aaeef9416/users/ad56afaeca0541848526a46133c496bb/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:46.248968 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41047565-3e56-4546-9700-82703005cdde admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ac14f3b99d4f44874e947aaeef9416, user_id=ad56afaeca0541848526a46133c496bb, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:46.251631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41047565-3e56-4546-9700-82703005cdde admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:46.267225 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1250/2507] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:06:46 2022] PUT /identity/v3/projects/b5ac14f3b99d4f44874e947aaeef9416/users/ad56afaeca0541848526a46133c496bb/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:06:46.279875 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf581c-c9b4-4393-a340-6c8dda3c978c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:46.280346 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf581c-c9b4-4393-a340-6c8dda3c978c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:46.280694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf581c-c9b4-4393-a340-6c8dda3c978c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:46.312810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e9cf581c-c9b4-4393-a340-6c8dda3c978c None None] MFA Rules not processed for user `ad56afaeca0541848526a46133c496bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:06:46.314315 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e9cf581c-c9b4-4393-a340-6c8dda3c978c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:46.384980 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1258/2508] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:06:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:06:46.389685 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41047565-3e56-4546-9700-82703005cdde admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:46.404609 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:46.460075 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41047565-3e56-4546-9700-82703005cdde admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:46.464994 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838adcf4-a1b2-4ad2-a70a-5fc2e95c8180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:46.465614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838adcf4-a1b2-4ad2-a70a-5fc2e95c8180 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:46.465899 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838adcf4-a1b2-4ad2-a70a-5fc2e95c8180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:46.466178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838adcf4-a1b2-4ad2-a70a-5fc2e95c8180 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:46.466815 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838adcf4-a1b2-4ad2-a70a-5fc2e95c8180 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:46.468744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838adcf4-a1b2-4ad2-a70a-5fc2e95c8180 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:46.496817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1251/2509] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:06:46 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:06:46.501597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf581c-c9b4-4393-a340-6c8dda3c978c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:46.513610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf581c-c9b4-4393-a340-6c8dda3c978c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:46.572493 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57468d-5508-4a6f-ae2c-02699fc94ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:46.573111 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57468d-5508-4a6f-ae2c-02699fc94ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:46.573390 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57468d-5508-4a6f-ae2c-02699fc94ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:46.573658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57468d-5508-4a6f-ae2c-02699fc94ee9 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:46.574282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57468d-5508-4a6f-ae2c-02699fc94ee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:46.576217 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57468d-5508-4a6f-ae2c-02699fc94ee9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:46.597588 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1259/2510] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:06:46 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:06:46.601879 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838adcf4-a1b2-4ad2-a70a-5fc2e95c8180 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:46.612592 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838adcf4-a1b2-4ad2-a70a-5fc2e95c8180 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:46.618086 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8438e-b9e6-46bd-aec1-02673e97a7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:46.618695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8438e-b9e6-46bd-aec1-02673e97a7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:46.618972 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8438e-b9e6-46bd-aec1-02673e97a7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:46.619248 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8438e-b9e6-46bd-aec1-02673e97a7e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:46.619892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8438e-b9e6-46bd-aec1-02673e97a7e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:46.621564 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8438e-b9e6-46bd-aec1-02673e97a7e9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:46.634215 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1252/2511] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:06:46.638553 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57468d-5508-4a6f-ae2c-02699fc94ee9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:46.651400 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57468d-5508-4a6f-ae2c-02699fc94ee9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:46.656220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586960c3-79c8-4566-89ba-1b63f4eff998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:46.656884 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586960c3-79c8-4566-89ba-1b63f4eff998 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:46.657168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586960c3-79c8-4566-89ba-1b63f4eff998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:46.657445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586960c3-79c8-4566-89ba-1b63f4eff998 admin admin] PATH_INFO: `/v3/projects/9031171b43d24d0380c9e756485f19f3/users/b021cda2e33245e8abe1bc1d8a1fe1ec/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:46.672331 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586960c3-79c8-4566-89ba-1b63f4eff998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9031171b43d24d0380c9e756485f19f3, user_id=b021cda2e33245e8abe1bc1d8a1fe1ec, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:46.674997 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586960c3-79c8-4566-89ba-1b63f4eff998 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:46.688830 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1260/2512] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:06:46 2022] PUT /identity/v3/projects/9031171b43d24d0380c9e756485f19f3/users/b021cda2e33245e8abe1bc1d8a1fe1ec/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:06:46.693689 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8438e-b9e6-46bd-aec1-02673e97a7e9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:46.703633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8438e-b9e6-46bd-aec1-02673e97a7e9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:46.770328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc31184-c431-47cc-8579-59052589e82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:46.770952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc31184-c431-47cc-8579-59052589e82c admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:46.771226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc31184-c431-47cc-8579-59052589e82c admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:46.771597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc31184-c431-47cc-8579-59052589e82c admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:46.772274 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc31184-c431-47cc-8579-59052589e82c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:46.773894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc31184-c431-47cc-8579-59052589e82c admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:46.783544 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1253/2513] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:06:46.788020 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586960c3-79c8-4566-89ba-1b63f4eff998 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:46.798340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586960c3-79c8-4566-89ba-1b63f4eff998 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:46.804373 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522acae-412a-49ea-bb0c-fbac16a1fa82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:46.805023 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522acae-412a-49ea-bb0c-fbac16a1fa82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:46.805299 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522acae-412a-49ea-bb0c-fbac16a1fa82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:46.805570 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522acae-412a-49ea-bb0c-fbac16a1fa82 admin admin] PATH_INFO: `/v3/projects/9031171b43d24d0380c9e756485f19f3/users/b021cda2e33245e8abe1bc1d8a1fe1ec/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:46.807496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522acae-412a-49ea-bb0c-fbac16a1fa82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9031171b43d24d0380c9e756485f19f3, user_id=b021cda2e33245e8abe1bc1d8a1fe1ec, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:46.810200 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522acae-412a-49ea-bb0c-fbac16a1fa82 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:46.812425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:46.823152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1261/2514] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:06:46 2022] PUT /identity/v3/projects/9031171b43d24d0380c9e756485f19f3/users/b021cda2e33245e8abe1bc1d8a1fe1ec/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:06:46.833504 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ff91c-e85a-4fc2-bdc9-2e8fc84ec8c7 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:46.833794 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ff91c-e85a-4fc2-bdc9-2e8fc84ec8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:46.834066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ff91c-e85a-4fc2-bdc9-2e8fc84ec8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:46.860716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-ef7ff91c-e85a-4fc2-bdc9-2e8fc84ec8c7 None None] MFA Rules not processed for user `b021cda2e33245e8abe1bc1d8a1fe1ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:06:46.861892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ef7ff91c-e85a-4fc2-bdc9-2e8fc84ec8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:46.932416 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1254/2515] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:06:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:06:46.944213 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522acae-412a-49ea-bb0c-fbac16a1fa82 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:46.946055 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1522acae-412a-49ea-bb0c-fbac16a1fa82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:06:47.005749 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522acae-412a-49ea-bb0c-fbac16a1fa82 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:47.009318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c771fe-821a-4ae9-bb30-d8af5ec001c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:47.009919 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c771fe-821a-4ae9-bb30-d8af5ec001c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:47.010195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c771fe-821a-4ae9-bb30-d8af5ec001c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:47.010461 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c771fe-821a-4ae9-bb30-d8af5ec001c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:47.019524 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c771fe-821a-4ae9-bb30-d8af5ec001c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:47.021012 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c771fe-821a-4ae9-bb30-d8af5ec001c3 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:47.030302 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1262/2516] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:47.105654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ff91c-e85a-4fc2-bdc9-2e8fc84ec8c7 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:06:47.116222 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ff91c-e85a-4fc2-bdc9-2e8fc84ec8c7 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:06:47.120039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025dc26-5b88-4133-8e9b-8957fa338efe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:06:47.120704 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025dc26-5b88-4133-8e9b-8957fa338efe service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:47.121138 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025dc26-5b88-4133-8e9b-8957fa338efe service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:47.121552 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025dc26-5b88-4133-8e9b-8957fa338efe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:47.126678 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:47.183834 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025dc26-5b88-4133-8e9b-8957fa338efe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:06:47.185300 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025dc26-5b88-4133-8e9b-8957fa338efe service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:06:47.195579 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1255/2517] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:06:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:06:57.026323 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:06:57.191057 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=25 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:07.931856 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c771fe-821a-4ae9-bb30-d8af5ec001c3 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:07.933289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b1c771fe-821a-4ae9-bb30-d8af5ec001c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:07.995956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c771fe-821a-4ae9-bb30-d8af5ec001c3 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:07.999957 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f3f1c-25f8-4ed6-a33a-2b356dedfda5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:08.001143 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f3f1c-25f8-4ed6-a33a-2b356dedfda5 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:08.001579 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f3f1c-25f8-4ed6-a33a-2b356dedfda5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:08.001853 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f3f1c-25f8-4ed6-a33a-2b356dedfda5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:08.003973 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ed8f3f1c-25f8-4ed6-a33a-2b356dedfda5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:08.073287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f3f1c-25f8-4ed6-a33a-2b356dedfda5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:08.074800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f3f1c-25f8-4ed6-a33a-2b356dedfda5 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:08.086551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1263/2518] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:15.186750 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025dc26-5b88-4133-8e9b-8957fa338efe service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:15.188466 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d025dc26-5b88-4133-8e9b-8957fa338efe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:15.200274 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025dc26-5b88-4133-8e9b-8957fa338efe service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:15.204475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d97c0-0a46-472b-990c-cd8962246228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:15.205274 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d97c0-0a46-472b-990c-cd8962246228 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:15.205425 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d97c0-0a46-472b-990c-cd8962246228 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:15.205633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d97c0-0a46-472b-990c-cd8962246228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:15.207340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4f7d97c0-0a46-472b-990c-cd8962246228 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:15.271647 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d97c0-0a46-472b-990c-cd8962246228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:15.273039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d97c0-0a46-472b-990c-cd8962246228 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:15.283566 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1256/2519] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:18.080961 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=48 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:18.550368 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f3f1c-25f8-4ed6-a33a-2b356dedfda5 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:18.551931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ed8f3f1c-25f8-4ed6-a33a-2b356dedfda5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:18.651315 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f3f1c-25f8-4ed6-a33a-2b356dedfda5 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:18.657334 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0ebdf-c81e-4ff9-8c0f-d4457248232f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33a0844a1c8f4ea4932e12d53e5ee9ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:18.657954 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0ebdf-c81e-4ff9-8c0f-d4457248232f service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:18.658235 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0ebdf-c81e-4ff9-8c0f-d4457248232f service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:18.658509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0ebdf-c81e-4ff9-8c0f-d4457248232f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:18.660345 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c3a0ebdf-c81e-4ff9-8c0f-d4457248232f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:18.672776 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0ebdf-c81e-4ff9-8c0f-d4457248232f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:18.674376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0ebdf-c81e-4ff9-8c0f-d4457248232f service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:18.685191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1264/2520] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:21.865455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d97c0-0a46-472b-990c-cd8962246228 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:21.866923 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4f7d97c0-0a46-472b-990c-cd8962246228 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:21.879897 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d97c0-0a46-472b-990c-cd8962246228 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:21.884559 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20681867-0253-4097-a0f0-5f82dd9627cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:21.885187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20681867-0253-4097-a0f0-5f82dd9627cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:21.885469 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20681867-0253-4097-a0f0-5f82dd9627cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:21.885763 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20681867-0253-4097-a0f0-5f82dd9627cc admin admin] PATH_INFO: `/v3/users/edc56fc598e741ceab98edd3b9afa28c` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:21.886964 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20681867-0253-4097-a0f0-5f82dd9627cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=edc56fc598e741ceab98edd3b9afa28c)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:21.888817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20681867-0253-4097-a0f0-5f82dd9627cc admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:21.937066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-20681867-0253-4097-a0f0-5f82dd9627cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edc56fc598e741ceab98edd3b9afa28c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:21.944285 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-20681867-0253-4097-a0f0-5f82dd9627cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edc56fc598e741ceab98edd3b9afa28c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:21.956158 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-20681867-0253-4097-a0f0-5f82dd9627cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edc56fc598e741ceab98edd3b9afa28c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:21.963077 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-20681867-0253-4097-a0f0-5f82dd9627cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edc56fc598e741ceab98edd3b9afa28c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:21.963927 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-20681867-0253-4097-a0f0-5f82dd9627cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edc56fc598e741ceab98edd3b9afa28c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:21.970339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1257/2521] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:07:21 2022] DELETE /identity/v3/users/edc56fc598e741ceab98edd3b9afa28c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:22.147992 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0ebdf-c81e-4ff9-8c0f-d4457248232f service placement] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:22.159243 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0ebdf-c81e-4ff9-8c0f-d4457248232f service placement] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:22.235836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99db746-2244-4df5-9cf0-cf8d12543d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:22.236551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99db746-2244-4df5-9cf0-cf8d12543d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:22.236829 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99db746-2244-4df5-9cf0-cf8d12543d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:22.237101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99db746-2244-4df5-9cf0-cf8d12543d6e admin admin] PATH_INFO: `/v3/projects/3d68ee6e33664bbfba3b239f16fbbaa0` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:22.238242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99db746-2244-4df5-9cf0-cf8d12543d6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d68ee6e33664bbfba3b239f16fbbaa0)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:22.240382 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99db746-2244-4df5-9cf0-cf8d12543d6e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:22.296536 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f99db746-2244-4df5-9cf0-cf8d12543d6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d68ee6e33664bbfba3b239f16fbbaa0', 'request_id': 'req-f99db746-2244-4df5-9cf0-cf8d12543d6e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:22.302477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f99db746-2244-4df5-9cf0-cf8d12543d6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d68ee6e33664bbfba3b239f16fbbaa0', 'request_id': 'req-f99db746-2244-4df5-9cf0-cf8d12543d6e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:22.310824 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f99db746-2244-4df5-9cf0-cf8d12543d6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d68ee6e33664bbfba3b239f16fbbaa0', 'request_id': 'req-f99db746-2244-4df5-9cf0-cf8d12543d6e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:22.322269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f99db746-2244-4df5-9cf0-cf8d12543d6e admin admin] The token cache is being invalidate because project 3d68ee6e33664bbfba3b239f16fbbaa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:07:22.322555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f99db746-2244-4df5-9cf0-cf8d12543d6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:22.323199 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f99db746-2244-4df5-9cf0-cf8d12543d6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:22.324957 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1265/2522] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:07:22 2022] DELETE /identity/v3/projects/3d68ee6e33664bbfba3b239f16fbbaa0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:22.329387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20681867-0253-4097-a0f0-5f82dd9627cc admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:22.331023 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-20681867-0253-4097-a0f0-5f82dd9627cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:22.345308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20681867-0253-4097-a0f0-5f82dd9627cc admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:22.404504 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd7be2-7f6e-4e24-a518-a2b5fec0a2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:22.405135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd7be2-7f6e-4e24-a518-a2b5fec0a2ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:22.405424 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd7be2-7f6e-4e24-a518-a2b5fec0a2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:22.405694 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd7be2-7f6e-4e24-a518-a2b5fec0a2ab admin admin] PATH_INFO: `/v3/users/bf53ca52f58d4170a256aae15fdaf8e7` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:22.406825 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd7be2-7f6e-4e24-a518-a2b5fec0a2ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf53ca52f58d4170a256aae15fdaf8e7)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:22.408872 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd7be2-7f6e-4e24-a518-a2b5fec0a2ab admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:22.414436 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:22.460710 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d2cd7be2-7f6e-4e24-a518-a2b5fec0a2ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf53ca52f58d4170a256aae15fdaf8e7'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:22.469095 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d2cd7be2-7f6e-4e24-a518-a2b5fec0a2ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf53ca52f58d4170a256aae15fdaf8e7'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:22.481643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d2cd7be2-7f6e-4e24-a518-a2b5fec0a2ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf53ca52f58d4170a256aae15fdaf8e7'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:22.488443 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d2cd7be2-7f6e-4e24-a518-a2b5fec0a2ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf53ca52f58d4170a256aae15fdaf8e7'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:22.489111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d2cd7be2-7f6e-4e24-a518-a2b5fec0a2ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf53ca52f58d4170a256aae15fdaf8e7'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:22.495227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1258/2523] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:07:22 2022] DELETE /identity/v3/users/bf53ca52f58d4170a256aae15fdaf8e7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:22.669301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99db746-2244-4df5-9cf0-cf8d12543d6e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:22.670762 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f99db746-2244-4df5-9cf0-cf8d12543d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:22.683592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99db746-2244-4df5-9cf0-cf8d12543d6e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:22.701730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:22.751580 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2b47d-f932-47f9-b4f0-6fe8a23a7646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:22.752342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2b47d-f932-47f9-b4f0-6fe8a23a7646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:22.752658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2b47d-f932-47f9-b4f0-6fe8a23a7646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:22.752986 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2b47d-f932-47f9-b4f0-6fe8a23a7646 admin admin] PATH_INFO: `/v3/projects/3ddab7b8618a42f1845f4ed86ca93b6b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:22.755082 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2b47d-f932-47f9-b4f0-6fe8a23a7646 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ddab7b8618a42f1845f4ed86ca93b6b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:22.757206 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2b47d-f932-47f9-b4f0-6fe8a23a7646 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:22.811006 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-22c2b47d-f932-47f9-b4f0-6fe8a23a7646 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ddab7b8618a42f1845f4ed86ca93b6b', 'request_id': 'req-22c2b47d-f932-47f9-b4f0-6fe8a23a7646'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:22.817528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-22c2b47d-f932-47f9-b4f0-6fe8a23a7646 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ddab7b8618a42f1845f4ed86ca93b6b', 'request_id': 'req-22c2b47d-f932-47f9-b4f0-6fe8a23a7646'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:22.826598 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-22c2b47d-f932-47f9-b4f0-6fe8a23a7646 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ddab7b8618a42f1845f4ed86ca93b6b', 'request_id': 'req-22c2b47d-f932-47f9-b4f0-6fe8a23a7646'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:22.853266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-22c2b47d-f932-47f9-b4f0-6fe8a23a7646 admin admin] The token cache is being invalidate because project 3ddab7b8618a42f1845f4ed86ca93b6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:07:22.853616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-22c2b47d-f932-47f9-b4f0-6fe8a23a7646 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:22.854244 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-22c2b47d-f932-47f9-b4f0-6fe8a23a7646 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:22.855936 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1266/2524] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:07:22 2022] DELETE /identity/v3/projects/3ddab7b8618a42f1845f4ed86ca93b6b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:32.491576 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:32.835357 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:45.956996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd7be2-7f6e-4e24-a518-a2b5fec0a2ab admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:45.958416 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d2cd7be2-7f6e-4e24-a518-a2b5fec0a2ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:45.972192 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd7be2-7f6e-4e24-a518-a2b5fec0a2ab admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:46.035697 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c553acf5-66c6-473f-88ee-1c01495f225d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:46.036367 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553acf5-66c6-473f-88ee-1c01495f225d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:46.036642 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553acf5-66c6-473f-88ee-1c01495f225d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:46.036910 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553acf5-66c6-473f-88ee-1c01495f225d admin admin] PATH_INFO: `/v3/users/a45d5f4ba3284323bb58bb90625ae94f` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.038020 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c553acf5-66c6-473f-88ee-1c01495f225d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a45d5f4ba3284323bb58bb90625ae94f)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:46.040133 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c553acf5-66c6-473f-88ee-1c01495f225d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:46.090517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c553acf5-66c6-473f-88ee-1c01495f225d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a45d5f4ba3284323bb58bb90625ae94f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:46.097268 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c553acf5-66c6-473f-88ee-1c01495f225d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a45d5f4ba3284323bb58bb90625ae94f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:46.111216 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c553acf5-66c6-473f-88ee-1c01495f225d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a45d5f4ba3284323bb58bb90625ae94f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:46.119069 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c553acf5-66c6-473f-88ee-1c01495f225d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a45d5f4ba3284323bb58bb90625ae94f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:46.119759 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c553acf5-66c6-473f-88ee-1c01495f225d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a45d5f4ba3284323bb58bb90625ae94f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:46.125674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1259/2525] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:07:45 2022] DELETE /identity/v3/users/a45d5f4ba3284323bb58bb90625ae94f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:46.304888 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2b47d-f932-47f9-b4f0-6fe8a23a7646 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:46.307049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-22c2b47d-f932-47f9-b4f0-6fe8a23a7646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:46.322857 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2b47d-f932-47f9-b4f0-6fe8a23a7646 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:46.380138 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb0914-8b4f-4b90-83e4-272582c37fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:46.380697 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb0914-8b4f-4b90-83e4-272582c37fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:46.380977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb0914-8b4f-4b90-83e4-272582c37fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:46.381251 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb0914-8b4f-4b90-83e4-272582c37fb6 admin admin] PATH_INFO: `/v3/projects/3e4a52a4c3744faca37f92db4493357a` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.382389 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb0914-8b4f-4b90-83e4-272582c37fb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e4a52a4c3744faca37f92db4493357a)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:46.384684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb0914-8b4f-4b90-83e4-272582c37fb6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:46.431893 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-08cb0914-8b4f-4b90-83e4-272582c37fb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e4a52a4c3744faca37f92db4493357a', 'request_id': 'req-08cb0914-8b4f-4b90-83e4-272582c37fb6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:46.437402 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-08cb0914-8b4f-4b90-83e4-272582c37fb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e4a52a4c3744faca37f92db4493357a', 'request_id': 'req-08cb0914-8b4f-4b90-83e4-272582c37fb6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:46.445039 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-08cb0914-8b4f-4b90-83e4-272582c37fb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e4a52a4c3744faca37f92db4493357a', 'request_id': 'req-08cb0914-8b4f-4b90-83e4-272582c37fb6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:46.456356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-08cb0914-8b4f-4b90-83e4-272582c37fb6 admin admin] The token cache is being invalidate because project 3e4a52a4c3744faca37f92db4493357a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:07:46.456618 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-08cb0914-8b4f-4b90-83e4-272582c37fb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:46.457254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-08cb0914-8b4f-4b90-83e4-272582c37fb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:46.458731 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1267/2526] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:07:46 2022] DELETE /identity/v3/projects/3e4a52a4c3744faca37f92db4493357a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:47.261330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c553acf5-66c6-473f-88ee-1c01495f225d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:47.262784 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c553acf5-66c6-473f-88ee-1c01495f225d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:47.275887 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c553acf5-66c6-473f-88ee-1c01495f225d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:47.332959 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0a894-0382-47f6-ae64-5a2540ae8409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:47.333516 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0a894-0382-47f6-ae64-5a2540ae8409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:47.333801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0a894-0382-47f6-ae64-5a2540ae8409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:47.334078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0a894-0382-47f6-ae64-5a2540ae8409 admin admin] PATH_INFO: `/v3/users/ad56afaeca0541848526a46133c496bb` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:47.335187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0a894-0382-47f6-ae64-5a2540ae8409 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad56afaeca0541848526a46133c496bb)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:47.337833 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0a894-0382-47f6-ae64-5a2540ae8409 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:47.360042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:47.386408 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a8f0a894-0382-47f6-ae64-5a2540ae8409 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad56afaeca0541848526a46133c496bb'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:47.394497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a8f0a894-0382-47f6-ae64-5a2540ae8409 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad56afaeca0541848526a46133c496bb'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:47.410005 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a8f0a894-0382-47f6-ae64-5a2540ae8409 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad56afaeca0541848526a46133c496bb'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:47.417670 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a8f0a894-0382-47f6-ae64-5a2540ae8409 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad56afaeca0541848526a46133c496bb'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:47.418408 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a8f0a894-0382-47f6-ae64-5a2540ae8409 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad56afaeca0541848526a46133c496bb'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:47.425942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1260/2527] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:07:47 2022] DELETE /identity/v3/users/ad56afaeca0541848526a46133c496bb => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:47.432025 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cf989-6b95-4f29-9d23-efcf534d84fa None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:47.432541 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cf989-6b95-4f29-9d23-efcf534d84fa None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:47.432639 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cf989-6b95-4f29-9d23-efcf534d84fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:47.451751 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d31cf989-6b95-4f29-9d23-efcf534d84fa None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:47.452984 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d31cf989-6b95-4f29-9d23-efcf534d84fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:47.512421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1268/2528] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:47.522288 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0a894-0382-47f6-ae64-5a2540ae8409 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:47.524332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a8f0a894-0382-47f6-ae64-5a2540ae8409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:47.545451 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31cf989-6b95-4f29-9d23-efcf534d84fa None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:47.547673 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d31cf989-6b95-4f29-9d23-efcf534d84fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:47.584605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:47.590356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0a894-0382-47f6-ae64-5a2540ae8409 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:47.595213 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308bc538-bf0e-4238-8e29-b350e4fb71e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:47.595826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bc538-bf0e-4238-8e29-b350e4fb71e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:47.596159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bc538-bf0e-4238-8e29-b350e4fb71e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:47.596465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bc538-bf0e-4238-8e29-b350e4fb71e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:47.606939 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308bc538-bf0e-4238-8e29-b350e4fb71e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:47.608456 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308bc538-bf0e-4238-8e29-b350e4fb71e3 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:47.619253 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1261/2529] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:47.627225 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31cf989-6b95-4f29-9d23-efcf534d84fa None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:47.632636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32787958-2504-4b95-8849-1a5898a42d11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:47.633788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32787958-2504-4b95-8849-1a5898a42d11 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:47.634052 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32787958-2504-4b95-8849-1a5898a42d11 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:47.634316 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32787958-2504-4b95-8849-1a5898a42d11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:47.636352 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-32787958-2504-4b95-8849-1a5898a42d11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:47.733690 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32787958-2504-4b95-8849-1a5898a42d11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:47.735345 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32787958-2504-4b95-8849-1a5898a42d11 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:47.757736 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1269/2530] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:47.912891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308bc538-bf0e-4238-8e29-b350e4fb71e3 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:47.915911 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-308bc538-bf0e-4238-8e29-b350e4fb71e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:47.933413 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308bc538-bf0e-4238-8e29-b350e4fb71e3 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:47.940506 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323bea7-1184-47a2-bfd2-644e5717b898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:47.941462 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323bea7-1184-47a2-bfd2-644e5717b898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:47.941803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323bea7-1184-47a2-bfd2-644e5717b898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:47.942269 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323bea7-1184-47a2-bfd2-644e5717b898 admin admin] PATH_INFO: `/v3/projects/b5ac14f3b99d4f44874e947aaeef9416` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:47.943918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323bea7-1184-47a2-bfd2-644e5717b898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5ac14f3b99d4f44874e947aaeef9416)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:47.946489 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323bea7-1184-47a2-bfd2-644e5717b898 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:48.001252 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4323bea7-1184-47a2-bfd2-644e5717b898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5ac14f3b99d4f44874e947aaeef9416', 'request_id': 'req-4323bea7-1184-47a2-bfd2-644e5717b898'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:48.009918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4323bea7-1184-47a2-bfd2-644e5717b898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5ac14f3b99d4f44874e947aaeef9416', 'request_id': 'req-4323bea7-1184-47a2-bfd2-644e5717b898'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:48.018732 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4323bea7-1184-47a2-bfd2-644e5717b898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5ac14f3b99d4f44874e947aaeef9416', 'request_id': 'req-4323bea7-1184-47a2-bfd2-644e5717b898'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:48.034060 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4323bea7-1184-47a2-bfd2-644e5717b898 admin admin] The token cache is being invalidate because project b5ac14f3b99d4f44874e947aaeef9416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:07:48.034415 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4323bea7-1184-47a2-bfd2-644e5717b898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:48.035391 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4323bea7-1184-47a2-bfd2-644e5717b898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:48.037603 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1262/2531] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:07:47 2022] DELETE /identity/v3/projects/b5ac14f3b99d4f44874e947aaeef9416 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:48.043343 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32787958-2504-4b95-8849-1a5898a42d11 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:48.060689 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32787958-2504-4b95-8849-1a5898a42d11 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:48.145581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b9287-bbed-4860-813f-158536cbd981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:48.146307 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b9287-bbed-4860-813f-158536cbd981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:48.146611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b9287-bbed-4860-813f-158536cbd981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:48.146981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b9287-bbed-4860-813f-158536cbd981 admin admin] PATH_INFO: `/v3/users/b021cda2e33245e8abe1bc1d8a1fe1ec` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:48.148288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b9287-bbed-4860-813f-158536cbd981 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b021cda2e33245e8abe1bc1d8a1fe1ec)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:48.151107 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b9287-bbed-4860-813f-158536cbd981 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:48.195061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:48.201476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a64b9287-bbed-4860-813f-158536cbd981 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b021cda2e33245e8abe1bc1d8a1fe1ec'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:48.202370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a64b9287-bbed-4860-813f-158536cbd981 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b021cda2e33245e8abe1bc1d8a1fe1ec'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:48.210045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a64b9287-bbed-4860-813f-158536cbd981 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b021cda2e33245e8abe1bc1d8a1fe1ec'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:48.217562 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a64b9287-bbed-4860-813f-158536cbd981 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b021cda2e33245e8abe1bc1d8a1fe1ec'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:48.229803 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a64b9287-bbed-4860-813f-158536cbd981 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b021cda2e33245e8abe1bc1d8a1fe1ec'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:48.235905 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1270/2532] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:07:48 2022] DELETE /identity/v3/users/b021cda2e33245e8abe1bc1d8a1fe1ec => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:48.510609 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323bea7-1184-47a2-bfd2-644e5717b898 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:48.512374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4323bea7-1184-47a2-bfd2-644e5717b898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:48.536049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323bea7-1184-47a2-bfd2-644e5717b898 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:48.558431 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:48.596758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697cda38-5633-43b4-83b0-92ebe693d4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:48.597572 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cda38-5633-43b4-83b0-92ebe693d4d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:48.597855 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cda38-5633-43b4-83b0-92ebe693d4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:48.598124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cda38-5633-43b4-83b0-92ebe693d4d4 admin admin] PATH_INFO: `/v3/projects/9031171b43d24d0380c9e756485f19f3` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:48.599238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697cda38-5633-43b4-83b0-92ebe693d4d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9031171b43d24d0380c9e756485f19f3)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:48.601200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697cda38-5633-43b4-83b0-92ebe693d4d4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:48.648560 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-697cda38-5633-43b4-83b0-92ebe693d4d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9031171b43d24d0380c9e756485f19f3', 'request_id': 'req-697cda38-5633-43b4-83b0-92ebe693d4d4'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:48.656973 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-697cda38-5633-43b4-83b0-92ebe693d4d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9031171b43d24d0380c9e756485f19f3', 'request_id': 'req-697cda38-5633-43b4-83b0-92ebe693d4d4'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:48.662862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-697cda38-5633-43b4-83b0-92ebe693d4d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9031171b43d24d0380c9e756485f19f3', 'request_id': 'req-697cda38-5633-43b4-83b0-92ebe693d4d4'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:48.674957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-697cda38-5633-43b4-83b0-92ebe693d4d4 admin admin] The token cache is being invalidate because project 9031171b43d24d0380c9e756485f19f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:07:48.675246 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-697cda38-5633-43b4-83b0-92ebe693d4d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:48.675889 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-697cda38-5633-43b4-83b0-92ebe693d4d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:48.677417 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1263/2533] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:07:48 2022] DELETE /identity/v3/projects/9031171b43d24d0380c9e756485f19f3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:48.683100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae833302-2f43-46d2-be5b-309e9cc8866a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:48.683629 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae833302-2f43-46d2-be5b-309e9cc8866a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:48.683729 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae833302-2f43-46d2-be5b-309e9cc8866a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:48.704933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ae833302-2f43-46d2-be5b-309e9cc8866a None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:48.706197 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ae833302-2f43-46d2-be5b-309e9cc8866a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:48.768909 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1271/2534] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:48.777574 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e528e0-07bd-4d96-98da-8a1c2a487b20 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:48.777907 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e528e0-07bd-4d96-98da-8a1c2a487b20 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:48.778159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e528e0-07bd-4d96-98da-8a1c2a487b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:48.798762 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f7e528e0-07bd-4d96-98da-8a1c2a487b20 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:48.800068 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f7e528e0-07bd-4d96-98da-8a1c2a487b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:48.809958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1264/2535] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:48.814229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae833302-2f43-46d2-be5b-309e9cc8866a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:48.824731 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae833302-2f43-46d2-be5b-309e9cc8866a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:48.829473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5211adb-43d7-4e5b-9479-3b5b873274e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:48.830075 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5211adb-43d7-4e5b-9479-3b5b873274e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:48.830378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5211adb-43d7-4e5b-9479-3b5b873274e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:48.830645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5211adb-43d7-4e5b-9479-3b5b873274e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:48.831334 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5211adb-43d7-4e5b-9479-3b5b873274e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:48.831628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5211adb-43d7-4e5b-9479-3b5b873274e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:48.833274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5211adb-43d7-4e5b-9479-3b5b873274e0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:48.844880 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1272/2536] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:48.849244 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e528e0-07bd-4d96-98da-8a1c2a487b20 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:48.859430 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e528e0-07bd-4d96-98da-8a1c2a487b20 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:48.864649 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a547bf1-79ab-418c-83aa-b86af87e4ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:48.865250 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a547bf1-79ab-418c-83aa-b86af87e4ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:48.865529 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a547bf1-79ab-418c-83aa-b86af87e4ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:48.865799 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a547bf1-79ab-418c-83aa-b86af87e4ff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:48.866429 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a547bf1-79ab-418c-83aa-b86af87e4ff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:48.868896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a547bf1-79ab-418c-83aa-b86af87e4ff7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:48.896167 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1265/2537] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:07:48 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:48.900654 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5211adb-43d7-4e5b-9479-3b5b873274e0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:48.910811 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5211adb-43d7-4e5b-9479-3b5b873274e0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:48.966737 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2960f-9ada-4830-82f5-9d9a9a242dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:48.967345 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2960f-9ada-4830-82f5-9d9a9a242dbf admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:48.967645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2960f-9ada-4830-82f5-9d9a9a242dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:48.967909 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2960f-9ada-4830-82f5-9d9a9a242dbf admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:48.968550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2960f-9ada-4830-82f5-9d9a9a242dbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:48.970482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2960f-9ada-4830-82f5-9d9a9a242dbf admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:48.991586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1273/2538] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:07:48 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:48.996240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a547bf1-79ab-418c-83aa-b86af87e4ff7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:49.006550 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a547bf1-79ab-418c-83aa-b86af87e4ff7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:49.012353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de001130-9579-47f2-afed-53b2194450bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:49.012953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de001130-9579-47f2-afed-53b2194450bf admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:49.013263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de001130-9579-47f2-afed-53b2194450bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:49.013531 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de001130-9579-47f2-afed-53b2194450bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:49.014177 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de001130-9579-47f2-afed-53b2194450bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:49.015763 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de001130-9579-47f2-afed-53b2194450bf admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:49.026072 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1266/2539] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:49.029883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2960f-9ada-4830-82f5-9d9a9a242dbf admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:49.039932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2960f-9ada-4830-82f5-9d9a9a242dbf admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:49.044599 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c6e70-73da-4741-adc6-2495286f3fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:49.045194 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c6e70-73da-4741-adc6-2495286f3fef admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:49.045473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c6e70-73da-4741-adc6-2495286f3fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:49.045744 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c6e70-73da-4741-adc6-2495286f3fef admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:49.046394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c6e70-73da-4741-adc6-2495286f3fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:49.047995 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c6e70-73da-4741-adc6-2495286f3fef admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:49.066242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1274/2540] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:49.070803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de001130-9579-47f2-afed-53b2194450bf admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:49.081739 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de001130-9579-47f2-afed-53b2194450bf admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:49.087008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35589b-e4a5-4cde-93a5-d0a4fb73ca82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:49.087672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35589b-e4a5-4cde-93a5-d0a4fb73ca82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:49.087949 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35589b-e4a5-4cde-93a5-d0a4fb73ca82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:49.089081 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35589b-e4a5-4cde-93a5-d0a4fb73ca82 admin admin] PATH_INFO: `/v3/projects/cbede1a9cf624979821e2e8f69fbb04b/users/4565a75ca6884a699aed5603ba2ede8b/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:49.105454 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35589b-e4a5-4cde-93a5-d0a4fb73ca82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbede1a9cf624979821e2e8f69fbb04b, user_id=4565a75ca6884a699aed5603ba2ede8b, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:49.108119 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35589b-e4a5-4cde-93a5-d0a4fb73ca82 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:49.122329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1267/2541] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:07:49 2022] PUT /identity/v3/projects/cbede1a9cf624979821e2e8f69fbb04b/users/4565a75ca6884a699aed5603ba2ede8b/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:49.134771 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176becb9-2257-44fc-8952-738f2d912bbd None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:49.135055 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176becb9-2257-44fc-8952-738f2d912bbd None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:49.135325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176becb9-2257-44fc-8952-738f2d912bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:49.164436 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-176becb9-2257-44fc-8952-738f2d912bbd None None] MFA Rules not processed for user `4565a75ca6884a699aed5603ba2ede8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:49.165756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-176becb9-2257-44fc-8952-738f2d912bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:49.177649 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:49.239837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1275/2542] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:07:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:49.244937 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35589b-e4a5-4cde-93a5-d0a4fb73ca82 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:49.301222 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:49.315035 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35589b-e4a5-4cde-93a5-d0a4fb73ca82 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:49.320711 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fb6cf-65b7-4181-a90e-d72b5f55617d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:49.321393 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fb6cf-65b7-4181-a90e-d72b5f55617d admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:49.321770 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fb6cf-65b7-4181-a90e-d72b5f55617d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:49.322064 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fb6cf-65b7-4181-a90e-d72b5f55617d admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:49.322692 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2fb6cf-65b7-4181-a90e-d72b5f55617d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:49.324642 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2fb6cf-65b7-4181-a90e-d72b5f55617d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:49.354388 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1268/2543] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:07:49 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:49.359491 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176becb9-2257-44fc-8952-738f2d912bbd None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:49.371306 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176becb9-2257-44fc-8952-738f2d912bbd None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:49.442032 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe11ee-ae25-4052-b313-43992cc04811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:49.442661 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe11ee-ae25-4052-b313-43992cc04811 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:49.442940 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe11ee-ae25-4052-b313-43992cc04811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:49.443210 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe11ee-ae25-4052-b313-43992cc04811 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:49.443847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe11ee-ae25-4052-b313-43992cc04811 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:49.446355 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe11ee-ae25-4052-b313-43992cc04811 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:49.472511 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1276/2544] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:07:49 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:07:49.476507 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fb6cf-65b7-4181-a90e-d72b5f55617d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:49.487538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fb6cf-65b7-4181-a90e-d72b5f55617d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:49.492928 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be4fa2-5f8f-425a-8e8d-35a9f44efff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:49.493631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be4fa2-5f8f-425a-8e8d-35a9f44efff8 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:49.493889 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be4fa2-5f8f-425a-8e8d-35a9f44efff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:49.494159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be4fa2-5f8f-425a-8e8d-35a9f44efff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:49.494799 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be4fa2-5f8f-425a-8e8d-35a9f44efff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:49.496393 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be4fa2-5f8f-425a-8e8d-35a9f44efff8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:49.508318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1269/2545] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:49.512876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe11ee-ae25-4052-b313-43992cc04811 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:49.523648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe11ee-ae25-4052-b313-43992cc04811 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:49.528874 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37415d42-c689-45d1-a3e0-447e7eb6c08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:49.529692 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37415d42-c689-45d1-a3e0-447e7eb6c08a admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:49.529970 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37415d42-c689-45d1-a3e0-447e7eb6c08a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:49.530240 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37415d42-c689-45d1-a3e0-447e7eb6c08a admin admin] PATH_INFO: `/v3/projects/654daa5c9c294450a61211d6d7f068bd/users/948494fdc049479cbd56e2509b92383f/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:49.546016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37415d42-c689-45d1-a3e0-447e7eb6c08a admin admin] RBAC: Authorizing `identity:create_grant(project_id=654daa5c9c294450a61211d6d7f068bd, user_id=948494fdc049479cbd56e2509b92383f, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:49.548692 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37415d42-c689-45d1-a3e0-447e7eb6c08a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:49.562923 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1277/2546] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:07:49 2022] PUT /identity/v3/projects/654daa5c9c294450a61211d6d7f068bd/users/948494fdc049479cbd56e2509b92383f/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:07:49.566893 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be4fa2-5f8f-425a-8e8d-35a9f44efff8 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:49.577516 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be4fa2-5f8f-425a-8e8d-35a9f44efff8 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:49.635070 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162dd9bb-cf66-431e-9bfe-c47d6b2e85ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:49.635680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162dd9bb-cf66-431e-9bfe-c47d6b2e85ae admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:49.635960 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162dd9bb-cf66-431e-9bfe-c47d6b2e85ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:49.636253 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162dd9bb-cf66-431e-9bfe-c47d6b2e85ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:49.636892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162dd9bb-cf66-431e-9bfe-c47d6b2e85ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:49.638469 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162dd9bb-cf66-431e-9bfe-c47d6b2e85ae admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:49.648717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1270/2547] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:07:49.652956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37415d42-c689-45d1-a3e0-447e7eb6c08a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:49.662988 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37415d42-c689-45d1-a3e0-447e7eb6c08a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:49.667940 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7684b4-51bd-4661-8a38-cb2c01aaa35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:49.668648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7684b4-51bd-4661-8a38-cb2c01aaa35d admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:49.668933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7684b4-51bd-4661-8a38-cb2c01aaa35d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:49.669208 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7684b4-51bd-4661-8a38-cb2c01aaa35d admin admin] PATH_INFO: `/v3/projects/654daa5c9c294450a61211d6d7f068bd/users/948494fdc049479cbd56e2509b92383f/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:49.671207 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7684b4-51bd-4661-8a38-cb2c01aaa35d admin admin] RBAC: Authorizing `identity:create_grant(project_id=654daa5c9c294450a61211d6d7f068bd, user_id=948494fdc049479cbd56e2509b92383f, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:49.674740 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7684b4-51bd-4661-8a38-cb2c01aaa35d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:49.687411 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1278/2548] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:07:49 2022] PUT /identity/v3/projects/654daa5c9c294450a61211d6d7f068bd/users/948494fdc049479cbd56e2509b92383f/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:49.698461 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9754d5-7ea7-4cfb-8487-0578bdd6b6e7 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:49.698670 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9754d5-7ea7-4cfb-8487-0578bdd6b6e7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:49.698941 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9754d5-7ea7-4cfb-8487-0578bdd6b6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:49.726700 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-2a9754d5-7ea7-4cfb-8487-0578bdd6b6e7 None None] MFA Rules not processed for user `948494fdc049479cbd56e2509b92383f`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:49.728013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2a9754d5-7ea7-4cfb-8487-0578bdd6b6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:49.799310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1271/2549] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:07:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:49.811570 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7684b4-51bd-4661-8a38-cb2c01aaa35d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:49.813399 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1f7684b4-51bd-4661-8a38-cb2c01aaa35d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:49.871736 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7684b4-51bd-4661-8a38-cb2c01aaa35d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:49.875576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9348890-3871-490b-8a9e-0e8cb960c6a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:49.876272 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9348890-3871-490b-8a9e-0e8cb960c6a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:49.876559 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9348890-3871-490b-8a9e-0e8cb960c6a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:49.876832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9348890-3871-490b-8a9e-0e8cb960c6a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:49.886549 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9348890-3871-490b-8a9e-0e8cb960c6a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:49.887981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9348890-3871-490b-8a9e-0e8cb960c6a2 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:49.898318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1279/2550] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:49.964630 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9754d5-7ea7-4cfb-8487-0578bdd6b6e7 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:50.015043 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:50.030520 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9754d5-7ea7-4cfb-8487-0578bdd6b6e7 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:50.035061 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc11b9c-9054-4108-a265-79899301e3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:50.035696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc11b9c-9054-4108-a265-79899301e3fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:50.035975 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc11b9c-9054-4108-a265-79899301e3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:50.036284 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc11b9c-9054-4108-a265-79899301e3fc admin admin] PATH_INFO: `/v3/users/4565a75ca6884a699aed5603ba2ede8b` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:50.037408 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc11b9c-9054-4108-a265-79899301e3fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4565a75ca6884a699aed5603ba2ede8b)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:50.039906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc11b9c-9054-4108-a265-79899301e3fc admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:50.085196 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1fc11b9c-9054-4108-a265-79899301e3fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4565a75ca6884a699aed5603ba2ede8b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:50.092513 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1fc11b9c-9054-4108-a265-79899301e3fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4565a75ca6884a699aed5603ba2ede8b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:50.105084 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1fc11b9c-9054-4108-a265-79899301e3fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4565a75ca6884a699aed5603ba2ede8b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:50.112173 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1fc11b9c-9054-4108-a265-79899301e3fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4565a75ca6884a699aed5603ba2ede8b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:50.112832 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-1fc11b9c-9054-4108-a265-79899301e3fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4565a75ca6884a699aed5603ba2ede8b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:50.119840 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1272/2551] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:07:49 2022] DELETE /identity/v3/users/4565a75ca6884a699aed5603ba2ede8b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:50.124522 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a3a4b-a588-41a6-b032-b5bcf9962355 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:50.124808 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a3a4b-a588-41a6-b032-b5bcf9962355 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:50.125078 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a3a4b-a588-41a6-b032-b5bcf9962355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:50.147041 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-952a3a4b-a588-41a6-b032-b5bcf9962355 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:50.148319 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-952a3a4b-a588-41a6-b032-b5bcf9962355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:50.152266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:50.216311 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1280/2552] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:50.227083 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc11b9c-9054-4108-a265-79899301e3fc admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:50.228565 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1fc11b9c-9054-4108-a265-79899301e3fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:50.292475 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc11b9c-9054-4108-a265-79899301e3fc admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:50.296220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c8a7b-19e1-43d7-ab1d-f3f424426971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:50.296833 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c8a7b-19e1-43d7-ab1d-f3f424426971 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:50.297121 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c8a7b-19e1-43d7-ab1d-f3f424426971 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:50.297387 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c8a7b-19e1-43d7-ab1d-f3f424426971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:50.306739 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c8a7b-19e1-43d7-ab1d-f3f424426971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:50.308194 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c8a7b-19e1-43d7-ab1d-f3f424426971 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:50.317705 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1273/2553] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:50.585824 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a3a4b-a588-41a6-b032-b5bcf9962355 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:50.587731 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-952a3a4b-a588-41a6-b032-b5bcf9962355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:50.607700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a3a4b-a588-41a6-b032-b5bcf9962355 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:50.613840 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33753c33-75ef-4b1f-a40e-302f7914c2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:50.614470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33753c33-75ef-4b1f-a40e-302f7914c2be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:50.614750 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33753c33-75ef-4b1f-a40e-302f7914c2be admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:50.615015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33753c33-75ef-4b1f-a40e-302f7914c2be admin admin] PATH_INFO: `/v3/projects/cbede1a9cf624979821e2e8f69fbb04b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:50.617187 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33753c33-75ef-4b1f-a40e-302f7914c2be admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbede1a9cf624979821e2e8f69fbb04b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:50.619121 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33753c33-75ef-4b1f-a40e-302f7914c2be admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:50.671703 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-33753c33-75ef-4b1f-a40e-302f7914c2be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbede1a9cf624979821e2e8f69fbb04b', 'request_id': 'req-33753c33-75ef-4b1f-a40e-302f7914c2be'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:50.677525 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-33753c33-75ef-4b1f-a40e-302f7914c2be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbede1a9cf624979821e2e8f69fbb04b', 'request_id': 'req-33753c33-75ef-4b1f-a40e-302f7914c2be'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:50.686044 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-33753c33-75ef-4b1f-a40e-302f7914c2be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbede1a9cf624979821e2e8f69fbb04b', 'request_id': 'req-33753c33-75ef-4b1f-a40e-302f7914c2be'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:50.698166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-33753c33-75ef-4b1f-a40e-302f7914c2be admin admin] The token cache is being invalidate because project cbede1a9cf624979821e2e8f69fbb04b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:07:50.698783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-33753c33-75ef-4b1f-a40e-302f7914c2be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:50.699235 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-33753c33-75ef-4b1f-a40e-302f7914c2be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:50.700758 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1281/2554] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:07:50 2022] DELETE /identity/v3/projects/cbede1a9cf624979821e2e8f69fbb04b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:50.704891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c8a7b-19e1-43d7-ab1d-f3f424426971 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:50.720241 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c8a7b-19e1-43d7-ab1d-f3f424426971 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:50.779894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06709e6-91a8-4a18-89d0-1c36c77ba29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:50.780568 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06709e6-91a8-4a18-89d0-1c36c77ba29b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:50.780850 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06709e6-91a8-4a18-89d0-1c36c77ba29b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:50.781125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06709e6-91a8-4a18-89d0-1c36c77ba29b admin admin] PATH_INFO: `/v3/users/948494fdc049479cbd56e2509b92383f` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:50.782238 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06709e6-91a8-4a18-89d0-1c36c77ba29b admin admin] RBAC: Authorizing `identity:delete_user(user_id=948494fdc049479cbd56e2509b92383f)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:50.784218 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06709e6-91a8-4a18-89d0-1c36c77ba29b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:50.794489 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:50.831703 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c06709e6-91a8-4a18-89d0-1c36c77ba29b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '948494fdc049479cbd56e2509b92383f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:50.840388 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c06709e6-91a8-4a18-89d0-1c36c77ba29b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '948494fdc049479cbd56e2509b92383f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:50.852465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c06709e6-91a8-4a18-89d0-1c36c77ba29b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '948494fdc049479cbd56e2509b92383f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:50.859206 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c06709e6-91a8-4a18-89d0-1c36c77ba29b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '948494fdc049479cbd56e2509b92383f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:50.859866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-c06709e6-91a8-4a18-89d0-1c36c77ba29b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '948494fdc049479cbd56e2509b92383f'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:50.865998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1274/2555] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:07:50 2022] DELETE /identity/v3/users/948494fdc049479cbd56e2509b92383f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:51.157394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33753c33-75ef-4b1f-a40e-302f7914c2be admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:51.158901 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-33753c33-75ef-4b1f-a40e-302f7914c2be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:51.171839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33753c33-75ef-4b1f-a40e-302f7914c2be admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:51.231140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f30c7-b51f-487c-8a2c-ce5a2cddb648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:51.231690 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f30c7-b51f-487c-8a2c-ce5a2cddb648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:51.231970 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f30c7-b51f-487c-8a2c-ce5a2cddb648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:51.232287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f30c7-b51f-487c-8a2c-ce5a2cddb648 admin admin] PATH_INFO: `/v3/projects/654daa5c9c294450a61211d6d7f068bd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:51.233501 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f30c7-b51f-487c-8a2c-ce5a2cddb648 admin admin] RBAC: Authorizing `identity:delete_project(project_id=654daa5c9c294450a61211d6d7f068bd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:51.235417 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f30c7-b51f-487c-8a2c-ce5a2cddb648 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:51.274929 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:51.286142 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-245f30c7-b51f-487c-8a2c-ce5a2cddb648 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '654daa5c9c294450a61211d6d7f068bd', 'request_id': 'req-245f30c7-b51f-487c-8a2c-ce5a2cddb648'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:51.292029 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-245f30c7-b51f-487c-8a2c-ce5a2cddb648 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '654daa5c9c294450a61211d6d7f068bd', 'request_id': 'req-245f30c7-b51f-487c-8a2c-ce5a2cddb648'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:51.299616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-245f30c7-b51f-487c-8a2c-ce5a2cddb648 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '654daa5c9c294450a61211d6d7f068bd', 'request_id': 'req-245f30c7-b51f-487c-8a2c-ce5a2cddb648'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:51.311312 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-245f30c7-b51f-487c-8a2c-ce5a2cddb648 admin admin] The token cache is being invalidate because project 654daa5c9c294450a61211d6d7f068bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:07:51.311596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-245f30c7-b51f-487c-8a2c-ce5a2cddb648 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:51.312272 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-245f30c7-b51f-487c-8a2c-ce5a2cddb648 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:51.313763 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1282/2556] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:07:51 2022] DELETE /identity/v3/projects/654daa5c9c294450a61211d6d7f068bd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:07:51.318590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76723a28-75a2-468d-a6b6-c62e2eebfa82 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:51.318876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76723a28-75a2-468d-a6b6-c62e2eebfa82 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:51.319148 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76723a28-75a2-468d-a6b6-c62e2eebfa82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:51.339893 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-76723a28-75a2-468d-a6b6-c62e2eebfa82 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:51.341125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-76723a28-75a2-468d-a6b6-c62e2eebfa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:51.401882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1275/2557] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:51.409762 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5371b-050a-433a-ae20-f27b66b02283 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:51.410050 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5371b-050a-433a-ae20-f27b66b02283 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:51.410325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5371b-050a-433a-ae20-f27b66b02283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:51.431533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ddb5371b-050a-433a-ae20-f27b66b02283 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:51.432792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ddb5371b-050a-433a-ae20-f27b66b02283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:51.441073 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1283/2558] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:51.446222 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76723a28-75a2-468d-a6b6-c62e2eebfa82 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:51.459892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76723a28-75a2-468d-a6b6-c62e2eebfa82 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:51.465247 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef3237-660c-4000-b05b-f496c8389e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:51.465859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef3237-660c-4000-b05b-f496c8389e7e admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:51.466157 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef3237-660c-4000-b05b-f496c8389e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:51.466431 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef3237-660c-4000-b05b-f496c8389e7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:51.467093 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef3237-660c-4000-b05b-f496c8389e7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:51.467386 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef3237-660c-4000-b05b-f496c8389e7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:51.469055 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef3237-660c-4000-b05b-f496c8389e7e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:51.481590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1276/2559] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:51.485854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5371b-050a-433a-ae20-f27b66b02283 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:51.496062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5371b-050a-433a-ae20-f27b66b02283 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:51.500809 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b732ce-0ea4-4ed8-aa50-185cb6218f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:51.501421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b732ce-0ea4-4ed8-aa50-185cb6218f6d admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:51.501683 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b732ce-0ea4-4ed8-aa50-185cb6218f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:51.501953 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b732ce-0ea4-4ed8-aa50-185cb6218f6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:51.502581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b732ce-0ea4-4ed8-aa50-185cb6218f6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:51.504517 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b732ce-0ea4-4ed8-aa50-185cb6218f6d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:51.531169 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1284/2560] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:07:51 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:51.535349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef3237-660c-4000-b05b-f496c8389e7e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:51.546118 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef3237-660c-4000-b05b-f496c8389e7e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:51.603112 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504cd430-744f-4bda-8316-e89be2074e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:51.603732 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cd430-744f-4bda-8316-e89be2074e49 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:51.604011 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cd430-744f-4bda-8316-e89be2074e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:51.604310 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cd430-744f-4bda-8316-e89be2074e49 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:51.604942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504cd430-744f-4bda-8316-e89be2074e49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:51.606860 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504cd430-744f-4bda-8316-e89be2074e49 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:51.635965 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1277/2561] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:07:51 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:51.640219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b732ce-0ea4-4ed8-aa50-185cb6218f6d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:51.651047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b732ce-0ea4-4ed8-aa50-185cb6218f6d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:51.655839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1cea5-95b1-44ef-b90f-b697a2ff5aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:51.656490 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1cea5-95b1-44ef-b90f-b697a2ff5aeb admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:51.656756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1cea5-95b1-44ef-b90f-b697a2ff5aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:51.657027 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1cea5-95b1-44ef-b90f-b697a2ff5aeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:51.657734 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1cea5-95b1-44ef-b90f-b697a2ff5aeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:51.659309 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1cea5-95b1-44ef-b90f-b697a2ff5aeb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:51.673243 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1285/2562] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:51.677144 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504cd430-744f-4bda-8316-e89be2074e49 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:51.684347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:51.688131 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504cd430-744f-4bda-8316-e89be2074e49 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:51.694505 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb69e1-973f-4c42-9adf-5ab1c84f14b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:51.695351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb69e1-973f-4c42-9adf-5ab1c84f14b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:51.695834 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb69e1-973f-4c42-9adf-5ab1c84f14b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:51.696161 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb69e1-973f-4c42-9adf-5ab1c84f14b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:51.696955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb69e1-973f-4c42-9adf-5ab1c84f14b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:51.698729 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb69e1-973f-4c42-9adf-5ab1c84f14b4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:51.709495 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1278/2563] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:51.714167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1cea5-95b1-44ef-b90f-b697a2ff5aeb admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:51.724395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1cea5-95b1-44ef-b90f-b697a2ff5aeb admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:51.729663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd999b-91e1-40e8-958b-da620672ec4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:51.730321 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd999b-91e1-40e8-958b-da620672ec4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:51.730603 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd999b-91e1-40e8-958b-da620672ec4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:51.730880 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd999b-91e1-40e8-958b-da620672ec4f admin admin] PATH_INFO: `/v3/projects/52394e166cf04832a1e7a23ff86fb4a1/users/110b364d3b6f49cca419fe6b4ffdb151/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:51.745986 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd999b-91e1-40e8-958b-da620672ec4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=52394e166cf04832a1e7a23ff86fb4a1, user_id=110b364d3b6f49cca419fe6b4ffdb151, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:51.748696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd999b-91e1-40e8-958b-da620672ec4f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:51.764596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1286/2564] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:07:51 2022] PUT /identity/v3/projects/52394e166cf04832a1e7a23ff86fb4a1/users/110b364d3b6f49cca419fe6b4ffdb151/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:51.777466 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcdd79-9cc7-4c8b-a925-6a5965cef257 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:51.777753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcdd79-9cc7-4c8b-a925-6a5965cef257 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:51.778025 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcdd79-9cc7-4c8b-a925-6a5965cef257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:51.805757 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-2ddcdd79-9cc7-4c8b-a925-6a5965cef257 None None] MFA Rules not processed for user `110b364d3b6f49cca419fe6b4ffdb151`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:51.806965 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2ddcdd79-9cc7-4c8b-a925-6a5965cef257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:51.874124 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1279/2565] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:07:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:51.878786 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd999b-91e1-40e8-958b-da620672ec4f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:51.939811 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd999b-91e1-40e8-958b-da620672ec4f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:51.951238 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f88a4-40f9-48ce-aef4-f856f394514c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:51.951866 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f88a4-40f9-48ce-aef4-f856f394514c admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:51.952181 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f88a4-40f9-48ce-aef4-f856f394514c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:51.952633 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f88a4-40f9-48ce-aef4-f856f394514c admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:51.953825 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f88a4-40f9-48ce-aef4-f856f394514c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:51.956980 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f88a4-40f9-48ce-aef4-f856f394514c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:51.988607 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1287/2566] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:07:51 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:51.993145 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcdd79-9cc7-4c8b-a925-6a5965cef257 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:52.004908 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcdd79-9cc7-4c8b-a925-6a5965cef257 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:52.067614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccdc18-8fbb-4851-814d-ae6a687368a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:52.068253 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccdc18-8fbb-4851-814d-ae6a687368a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:52.068527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccdc18-8fbb-4851-814d-ae6a687368a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:52.068795 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccdc18-8fbb-4851-814d-ae6a687368a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:52.069450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeccdc18-8fbb-4851-814d-ae6a687368a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:52.071370 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeccdc18-8fbb-4851-814d-ae6a687368a9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:52.096760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1280/2567] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:07:51 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:52.101407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f88a4-40f9-48ce-aef4-f856f394514c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:52.114513 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f88a4-40f9-48ce-aef4-f856f394514c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:52.119430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bbd54-d951-4f47-bee4-4b0292d39d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:52.120038 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bbd54-d951-4f47-bee4-4b0292d39d07 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:52.120483 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bbd54-d951-4f47-bee4-4b0292d39d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:52.120748 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bbd54-d951-4f47-bee4-4b0292d39d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:52.121394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6bbd54-d951-4f47-bee4-4b0292d39d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:52.123005 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6bbd54-d951-4f47-bee4-4b0292d39d07 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:52.137164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1288/2568] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:52.141574 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccdc18-8fbb-4851-814d-ae6a687368a9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:52.155318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccdc18-8fbb-4851-814d-ae6a687368a9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:52.160321 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6d96c-5ca2-4c64-951f-6593342ae403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:52.161017 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6d96c-5ca2-4c64-951f-6593342ae403 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:52.161280 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6d96c-5ca2-4c64-951f-6593342ae403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:52.161636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6d96c-5ca2-4c64-951f-6593342ae403 admin admin] PATH_INFO: `/v3/projects/7120abef4df84f11ab730b0f3f2f32a1/users/0e589b6a3b97431c8c15eef8266c77f7/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:52.184327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6d96c-5ca2-4c64-951f-6593342ae403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7120abef4df84f11ab730b0f3f2f32a1, user_id=0e589b6a3b97431c8c15eef8266c77f7, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:52.186905 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6d96c-5ca2-4c64-951f-6593342ae403 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:52.205756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1281/2569] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:07:52 2022] PUT /identity/v3/projects/7120abef4df84f11ab730b0f3f2f32a1/users/0e589b6a3b97431c8c15eef8266c77f7/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:07:52.212791 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bbd54-d951-4f47-bee4-4b0292d39d07 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:52.223698 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bbd54-d951-4f47-bee4-4b0292d39d07 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:52.265630 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:52.299043 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f714bcd-d224-4b4c-a082-5a26eebdb206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:52.301837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f714bcd-d224-4b4c-a082-5a26eebdb206 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:52.302133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f714bcd-d224-4b4c-a082-5a26eebdb206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:52.302412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f714bcd-d224-4b4c-a082-5a26eebdb206 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:52.303071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f714bcd-d224-4b4c-a082-5a26eebdb206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:52.304841 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f714bcd-d224-4b4c-a082-5a26eebdb206 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:52.315913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1289/2570] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:07:52.321846 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6d96c-5ca2-4c64-951f-6593342ae403 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:52.336704 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6d96c-5ca2-4c64-951f-6593342ae403 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:52.341625 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e73a09-0bef-4286-af5c-08dc240c1dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:52.342304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e73a09-0bef-4286-af5c-08dc240c1dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:52.342586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e73a09-0bef-4286-af5c-08dc240c1dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:52.342930 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e73a09-0bef-4286-af5c-08dc240c1dc4 admin admin] PATH_INFO: `/v3/projects/7120abef4df84f11ab730b0f3f2f32a1/users/0e589b6a3b97431c8c15eef8266c77f7/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:52.345029 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e73a09-0bef-4286-af5c-08dc240c1dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7120abef4df84f11ab730b0f3f2f32a1, user_id=0e589b6a3b97431c8c15eef8266c77f7, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:52.348769 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e73a09-0bef-4286-af5c-08dc240c1dc4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:52.362251 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1282/2571] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:07:52 2022] PUT /identity/v3/projects/7120abef4df84f11ab730b0f3f2f32a1/users/0e589b6a3b97431c8c15eef8266c77f7/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:52.373043 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cef0b-56e0-42ee-b752-b1459ace2d13 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:52.373306 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cef0b-56e0-42ee-b752-b1459ace2d13 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:52.373609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cef0b-56e0-42ee-b752-b1459ace2d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:52.406668 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-883cef0b-56e0-42ee-b752-b1459ace2d13 None None] MFA Rules not processed for user `0e589b6a3b97431c8c15eef8266c77f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:52.407869 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-883cef0b-56e0-42ee-b752-b1459ace2d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:52.491798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1290/2572] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:07:52 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:52.506413 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e73a09-0bef-4286-af5c-08dc240c1dc4 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:52.509018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-c1e73a09-0bef-4286-af5c-08dc240c1dc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:52.556264 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:52.578216 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e73a09-0bef-4286-af5c-08dc240c1dc4 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:52.581966 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df55057-af10-4017-a64d-be1ffea25baa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:52.582578 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df55057-af10-4017-a64d-be1ffea25baa service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:52.582862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df55057-af10-4017-a64d-be1ffea25baa service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:52.583264 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df55057-af10-4017-a64d-be1ffea25baa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:52.593106 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df55057-af10-4017-a64d-be1ffea25baa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:52.594557 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df55057-af10-4017-a64d-be1ffea25baa service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:52.605560 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1283/2573] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:52 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:52.772176 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883cef0b-56e0-42ee-b752-b1459ace2d13 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:52.836859 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883cef0b-56e0-42ee-b752-b1459ace2d13 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:52.841542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122867a7-cdc0-416a-809a-70338558cf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:52.842210 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122867a7-cdc0-416a-809a-70338558cf7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:52.842492 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122867a7-cdc0-416a-809a-70338558cf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:52.842764 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122867a7-cdc0-416a-809a-70338558cf7b admin admin] PATH_INFO: `/v3/users/110b364d3b6f49cca419fe6b4ffdb151` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:52.843921 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122867a7-cdc0-416a-809a-70338558cf7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=110b364d3b6f49cca419fe6b4ffdb151)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:52.845926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122867a7-cdc0-416a-809a-70338558cf7b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:52.894234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-122867a7-cdc0-416a-809a-70338558cf7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '110b364d3b6f49cca419fe6b4ffdb151'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:52.894905 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-122867a7-cdc0-416a-809a-70338558cf7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '110b364d3b6f49cca419fe6b4ffdb151'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:52.902514 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-122867a7-cdc0-416a-809a-70338558cf7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '110b364d3b6f49cca419fe6b4ffdb151'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:52.905198 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:52.913464 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-122867a7-cdc0-416a-809a-70338558cf7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '110b364d3b6f49cca419fe6b4ffdb151'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:52.925987 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-122867a7-cdc0-416a-809a-70338558cf7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '110b364d3b6f49cca419fe6b4ffdb151'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:52.932723 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1291/2574] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:07:52 2022] DELETE /identity/v3/users/110b364d3b6f49cca419fe6b4ffdb151 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:52.937242 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f376d21-f759-4e0d-a836-233dc58b9299 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:52.937735 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f376d21-f759-4e0d-a836-233dc58b9299 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:52.937846 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f376d21-f759-4e0d-a836-233dc58b9299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:52.956990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-2f376d21-f759-4e0d-a836-233dc58b9299 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:52.958184 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2f376d21-f759-4e0d-a836-233dc58b9299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:53.018752 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1284/2575] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:07:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:53.029148 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122867a7-cdc0-416a-809a-70338558cf7b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:53.030564 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-122867a7-cdc0-416a-809a-70338558cf7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:53.089214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122867a7-cdc0-416a-809a-70338558cf7b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:53.092773 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acff1fe-aa87-498a-9296-1b07cb8751f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:53.093381 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acff1fe-aa87-498a-9296-1b07cb8751f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:53.093660 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acff1fe-aa87-498a-9296-1b07cb8751f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:53.093930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acff1fe-aa87-498a-9296-1b07cb8751f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:53.103347 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acff1fe-aa87-498a-9296-1b07cb8751f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:53.104807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acff1fe-aa87-498a-9296-1b07cb8751f5 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:53.117175 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1292/2576] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:53.390097 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f376d21-f759-4e0d-a836-233dc58b9299 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:53.391328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2f376d21-f759-4e0d-a836-233dc58b9299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:53.409110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f376d21-f759-4e0d-a836-233dc58b9299 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:53.415095 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44b037-02fa-4cb5-a461-8bb7eefd2d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:53.415751 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44b037-02fa-4cb5-a461-8bb7eefd2d74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:53.416161 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44b037-02fa-4cb5-a461-8bb7eefd2d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:53.416516 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44b037-02fa-4cb5-a461-8bb7eefd2d74 admin admin] PATH_INFO: `/v3/projects/52394e166cf04832a1e7a23ff86fb4a1` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:53.417512 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44b037-02fa-4cb5-a461-8bb7eefd2d74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52394e166cf04832a1e7a23ff86fb4a1)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:53.419425 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44b037-02fa-4cb5-a461-8bb7eefd2d74 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:53.467048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6e44b037-02fa-4cb5-a461-8bb7eefd2d74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52394e166cf04832a1e7a23ff86fb4a1', 'request_id': 'req-6e44b037-02fa-4cb5-a461-8bb7eefd2d74'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:53.474451 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6e44b037-02fa-4cb5-a461-8bb7eefd2d74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52394e166cf04832a1e7a23ff86fb4a1', 'request_id': 'req-6e44b037-02fa-4cb5-a461-8bb7eefd2d74'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:53.480688 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6e44b037-02fa-4cb5-a461-8bb7eefd2d74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52394e166cf04832a1e7a23ff86fb4a1', 'request_id': 'req-6e44b037-02fa-4cb5-a461-8bb7eefd2d74'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:53.492179 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6e44b037-02fa-4cb5-a461-8bb7eefd2d74 admin admin] The token cache is being invalidate because project 52394e166cf04832a1e7a23ff86fb4a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:07:53.492472 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6e44b037-02fa-4cb5-a461-8bb7eefd2d74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:53.493115 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6e44b037-02fa-4cb5-a461-8bb7eefd2d74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:53.494618 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1285/2577] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:07:53 2022] DELETE /identity/v3/projects/52394e166cf04832a1e7a23ff86fb4a1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:53.498598 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acff1fe-aa87-498a-9296-1b07cb8751f5 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:53.508510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acff1fe-aa87-498a-9296-1b07cb8751f5 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:53.571088 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7247574-74da-469b-890c-c9a6684f0f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:53.571608 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7247574-74da-469b-890c-c9a6684f0f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:53.574180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7247574-74da-469b-890c-c9a6684f0f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:53.574180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7247574-74da-469b-890c-c9a6684f0f15 admin admin] PATH_INFO: `/v3/users/0e589b6a3b97431c8c15eef8266c77f7` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:53.574180 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7247574-74da-469b-890c-c9a6684f0f15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e589b6a3b97431c8c15eef8266c77f7)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:53.575468 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7247574-74da-469b-890c-c9a6684f0f15 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:53.624258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e7247574-74da-469b-890c-c9a6684f0f15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e589b6a3b97431c8c15eef8266c77f7'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:53.624907 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e7247574-74da-469b-890c-c9a6684f0f15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e589b6a3b97431c8c15eef8266c77f7'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:53.632889 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e7247574-74da-469b-890c-c9a6684f0f15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e589b6a3b97431c8c15eef8266c77f7'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:53.639668 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e7247574-74da-469b-890c-c9a6684f0f15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e589b6a3b97431c8c15eef8266c77f7'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:53.650824 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:53.652082 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e7247574-74da-469b-890c-c9a6684f0f15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e589b6a3b97431c8c15eef8266c77f7'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:53.660120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1293/2578] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:07:53 2022] DELETE /identity/v3/users/0e589b6a3b97431c8c15eef8266c77f7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:53.933949 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44b037-02fa-4cb5-a461-8bb7eefd2d74 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:53.935554 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6e44b037-02fa-4cb5-a461-8bb7eefd2d74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:53.950884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44b037-02fa-4cb5-a461-8bb7eefd2d74 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:54.016781 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2bac7-61ad-4826-b2d6-d4a0ad0b5a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:54.017446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2bac7-61ad-4826-b2d6-d4a0ad0b5a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.017720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2bac7-61ad-4826-b2d6-d4a0ad0b5a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.018007 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2bac7-61ad-4826-b2d6-d4a0ad0b5a0b admin admin] PATH_INFO: `/v3/projects/7120abef4df84f11ab730b0f3f2f32a1` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.019087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2bac7-61ad-4826-b2d6-d4a0ad0b5a0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7120abef4df84f11ab730b0f3f2f32a1)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:54.021028 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2bac7-61ad-4826-b2d6-d4a0ad0b5a0b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:54.029058 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:54.072533 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ebc2bac7-61ad-4826-b2d6-d4a0ad0b5a0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7120abef4df84f11ab730b0f3f2f32a1', 'request_id': 'req-ebc2bac7-61ad-4826-b2d6-d4a0ad0b5a0b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:54.080606 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ebc2bac7-61ad-4826-b2d6-d4a0ad0b5a0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7120abef4df84f11ab730b0f3f2f32a1', 'request_id': 'req-ebc2bac7-61ad-4826-b2d6-d4a0ad0b5a0b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:54.086429 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ebc2bac7-61ad-4826-b2d6-d4a0ad0b5a0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7120abef4df84f11ab730b0f3f2f32a1', 'request_id': 'req-ebc2bac7-61ad-4826-b2d6-d4a0ad0b5a0b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:54.097572 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ebc2bac7-61ad-4826-b2d6-d4a0ad0b5a0b admin admin] The token cache is being invalidate because project 7120abef4df84f11ab730b0f3f2f32a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:07:54.097862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ebc2bac7-61ad-4826-b2d6-d4a0ad0b5a0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:54.098497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-ebc2bac7-61ad-4826-b2d6-d4a0ad0b5a0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:07:54.100457 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1286/2579] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:07:53 2022] DELETE /identity/v3/projects/7120abef4df84f11ab730b0f3f2f32a1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:54.105721 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671cae7-9297-4af3-91bf-35b02f62e116 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.106223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671cae7-9297-4af3-91bf-35b02f62e116 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.106352 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671cae7-9297-4af3-91bf-35b02f62e116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.126167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7671cae7-9297-4af3-91bf-35b02f62e116 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:54.127343 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7671cae7-9297-4af3-91bf-35b02f62e116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:54.190938 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1294/2580] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:54.199517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b36d8-c30e-4483-bdc1-091d93b7e345 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.199831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b36d8-c30e-4483-bdc1-091d93b7e345 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.200136 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b36d8-c30e-4483-bdc1-091d93b7e345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.219765 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-194b36d8-c30e-4483-bdc1-091d93b7e345 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:54.220999 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-194b36d8-c30e-4483-bdc1-091d93b7e345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:54.228918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1287/2581] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:54.233216 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671cae7-9297-4af3-91bf-35b02f62e116 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:54.243839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671cae7-9297-4af3-91bf-35b02f62e116 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:54.248514 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b92779-87a8-488e-a5dc-becee0248b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:54.249218 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b92779-87a8-488e-a5dc-becee0248b29 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.249487 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b92779-87a8-488e-a5dc-becee0248b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.249756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b92779-87a8-488e-a5dc-becee0248b29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.250426 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b92779-87a8-488e-a5dc-becee0248b29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:54.250727 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b92779-87a8-488e-a5dc-becee0248b29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:54.252337 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b92779-87a8-488e-a5dc-becee0248b29 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:54.264798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1295/2582] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:54.269098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b36d8-c30e-4483-bdc1-091d93b7e345 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:54.278621 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b36d8-c30e-4483-bdc1-091d93b7e345 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:54.283315 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204c0d8-8d56-4740-814a-8ca679b86391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:54.283938 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204c0d8-8d56-4740-814a-8ca679b86391 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.284259 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204c0d8-8d56-4740-814a-8ca679b86391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.284531 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204c0d8-8d56-4740-814a-8ca679b86391 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.285163 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8204c0d8-8d56-4740-814a-8ca679b86391 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:54.287086 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8204c0d8-8d56-4740-814a-8ca679b86391 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:54.314461 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1288/2583] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:07:54 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:54.318728 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b92779-87a8-488e-a5dc-becee0248b29 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:54.329178 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b92779-87a8-488e-a5dc-becee0248b29 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:54.386552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745c856-e677-40cc-9fbe-baff245ea841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:54.387161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745c856-e677-40cc-9fbe-baff245ea841 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.387443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745c856-e677-40cc-9fbe-baff245ea841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.387716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745c856-e677-40cc-9fbe-baff245ea841 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.388378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745c856-e677-40cc-9fbe-baff245ea841 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:54.390305 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745c856-e677-40cc-9fbe-baff245ea841 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:54.411816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1296/2584] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:07:54 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:54.416049 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204c0d8-8d56-4740-814a-8ca679b86391 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:54.426330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204c0d8-8d56-4740-814a-8ca679b86391 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:54.431842 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3dd38-75a5-4e58-aa1e-f62f79a0ad69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:54.432472 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3dd38-75a5-4e58-aa1e-f62f79a0ad69 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.432753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3dd38-75a5-4e58-aa1e-f62f79a0ad69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.433024 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3dd38-75a5-4e58-aa1e-f62f79a0ad69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.433666 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3dd38-75a5-4e58-aa1e-f62f79a0ad69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:54.435257 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3dd38-75a5-4e58-aa1e-f62f79a0ad69 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:54.445926 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1289/2585] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:54.450373 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745c856-e677-40cc-9fbe-baff245ea841 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:54.461336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745c856-e677-40cc-9fbe-baff245ea841 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:54.466074 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bc3c8-4a52-4226-9fb6-e822e9b21b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:54.466676 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bc3c8-4a52-4226-9fb6-e822e9b21b50 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.466985 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bc3c8-4a52-4226-9fb6-e822e9b21b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.467261 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bc3c8-4a52-4226-9fb6-e822e9b21b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.467907 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5bc3c8-4a52-4226-9fb6-e822e9b21b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:54.469529 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5bc3c8-4a52-4226-9fb6-e822e9b21b50 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:54.486047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1297/2586] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:54.491559 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3dd38-75a5-4e58-aa1e-f62f79a0ad69 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:54.501954 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3dd38-75a5-4e58-aa1e-f62f79a0ad69 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:54.506868 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6e59d-0774-47f8-99f3-c859fe9a5690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:54.507629 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6e59d-0774-47f8-99f3-c859fe9a5690 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.507913 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6e59d-0774-47f8-99f3-c859fe9a5690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.508221 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6e59d-0774-47f8-99f3-c859fe9a5690 admin admin] PATH_INFO: `/v3/projects/28fa44465115400d853799bb7589152b/users/54520805f705410983bfe514b73181e0/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.523093 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6e59d-0774-47f8-99f3-c859fe9a5690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28fa44465115400d853799bb7589152b, user_id=54520805f705410983bfe514b73181e0, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:54.525785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6e59d-0774-47f8-99f3-c859fe9a5690 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:54.539290 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1290/2587] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:07:54 2022] PUT /identity/v3/projects/28fa44465115400d853799bb7589152b/users/54520805f705410983bfe514b73181e0/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:54.550509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e949bc8-3081-40e7-927b-9dbc70a6f78d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.550832 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e949bc8-3081-40e7-927b-9dbc70a6f78d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.551108 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e949bc8-3081-40e7-927b-9dbc70a6f78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.579267 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6e949bc8-3081-40e7-927b-9dbc70a6f78d None None] MFA Rules not processed for user `54520805f705410983bfe514b73181e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:54.580510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6e949bc8-3081-40e7-927b-9dbc70a6f78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:54.628105 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:54.650821 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1298/2588] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:07:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:54.655334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6e59d-0774-47f8-99f3-c859fe9a5690 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:54.719150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6e59d-0774-47f8-99f3-c859fe9a5690 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:54.723978 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bcf2d-930d-4b4a-b147-d8eda5a2ca88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:54.724624 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bcf2d-930d-4b4a-b147-d8eda5a2ca88 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.724963 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bcf2d-930d-4b4a-b147-d8eda5a2ca88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.725773 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bcf2d-930d-4b4a-b147-d8eda5a2ca88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.726417 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bcf2d-930d-4b4a-b147-d8eda5a2ca88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:54.728422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bcf2d-930d-4b4a-b147-d8eda5a2ca88 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:54.757110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1291/2589] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:07:54 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:54.759774 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e949bc8-3081-40e7-927b-9dbc70a6f78d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:54.770321 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e949bc8-3081-40e7-927b-9dbc70a6f78d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:54.828740 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb528230-6e22-4ce7-93e7-f66d4f3ee602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:54.829354 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb528230-6e22-4ce7-93e7-f66d4f3ee602 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.829628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb528230-6e22-4ce7-93e7-f66d4f3ee602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.829949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb528230-6e22-4ce7-93e7-f66d4f3ee602 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.830583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb528230-6e22-4ce7-93e7-f66d4f3ee602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:54.832518 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb528230-6e22-4ce7-93e7-f66d4f3ee602 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:54.855011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1299/2590] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:07:54 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:07:54.860303 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bcf2d-930d-4b4a-b147-d8eda5a2ca88 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:54.871239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bcf2d-930d-4b4a-b147-d8eda5a2ca88 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:54.875961 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51aa51-8dff-46b8-b755-9e18bca3d5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:54.876628 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51aa51-8dff-46b8-b755-9e18bca3d5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.876912 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51aa51-8dff-46b8-b755-9e18bca3d5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.877187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51aa51-8dff-46b8-b755-9e18bca3d5a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.877845 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51aa51-8dff-46b8-b755-9e18bca3d5a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:54.879432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51aa51-8dff-46b8-b755-9e18bca3d5a1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:54.890073 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1292/2591] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:07:54.894388 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb528230-6e22-4ce7-93e7-f66d4f3ee602 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:54.905502 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb528230-6e22-4ce7-93e7-f66d4f3ee602 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:54.910346 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4f0c4-b2ca-462e-81d2-c634da23a407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:54.910990 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4f0c4-b2ca-462e-81d2-c634da23a407 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.911274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4f0c4-b2ca-462e-81d2-c634da23a407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.911551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4f0c4-b2ca-462e-81d2-c634da23a407 admin admin] PATH_INFO: `/v3/projects/950cdc8329004b7d83dfed87d8d92d79/users/4670fe3b25a240d3aa735c0d1ff1b249/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.926801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4f0c4-b2ca-462e-81d2-c634da23a407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=950cdc8329004b7d83dfed87d8d92d79, user_id=4670fe3b25a240d3aa735c0d1ff1b249, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:54.929871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4f0c4-b2ca-462e-81d2-c634da23a407 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:54.943031 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1300/2592] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:07:54 2022] PUT /identity/v3/projects/950cdc8329004b7d83dfed87d8d92d79/users/4670fe3b25a240d3aa735c0d1ff1b249/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:54.947328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51aa51-8dff-46b8-b755-9e18bca3d5a1 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:54.952455 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:54.958930 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51aa51-8dff-46b8-b755-9e18bca3d5a1 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:55.017187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e935d45-6dfa-42a7-96c6-4504b9647c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:55.017755 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e935d45-6dfa-42a7-96c6-4504b9647c3e admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:55.018033 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e935d45-6dfa-42a7-96c6-4504b9647c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:55.018304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e935d45-6dfa-42a7-96c6-4504b9647c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:55.018984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e935d45-6dfa-42a7-96c6-4504b9647c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:55.020638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e935d45-6dfa-42a7-96c6-4504b9647c3e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:55.031353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1293/2593] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:07:54 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:07:55.035786 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4f0c4-b2ca-462e-81d2-c634da23a407 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:55.047009 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4f0c4-b2ca-462e-81d2-c634da23a407 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:55.051725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a70f5-d88e-4a24-b27f-62d6cb0fe18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:55.052407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a70f5-d88e-4a24-b27f-62d6cb0fe18a admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:55.052696 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a70f5-d88e-4a24-b27f-62d6cb0fe18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:55.053104 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a70f5-d88e-4a24-b27f-62d6cb0fe18a admin admin] PATH_INFO: `/v3/projects/950cdc8329004b7d83dfed87d8d92d79/users/4670fe3b25a240d3aa735c0d1ff1b249/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:55.055028 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a70f5-d88e-4a24-b27f-62d6cb0fe18a admin admin] RBAC: Authorizing `identity:create_grant(project_id=950cdc8329004b7d83dfed87d8d92d79, user_id=4670fe3b25a240d3aa735c0d1ff1b249, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:55.057685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a70f5-d88e-4a24-b27f-62d6cb0fe18a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:55.067997 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1301/2594] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:07:55 2022] PUT /identity/v3/projects/950cdc8329004b7d83dfed87d8d92d79/users/4670fe3b25a240d3aa735c0d1ff1b249/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:55.078415 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1c6c1-2b1f-443e-b40a-14f345d0cdfd None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:55.078697 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1c6c1-2b1f-443e-b40a-14f345d0cdfd None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:55.078971 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1c6c1-2b1f-443e-b40a-14f345d0cdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:55.107388 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e6a1c6c1-2b1f-443e-b40a-14f345d0cdfd None None] MFA Rules not processed for user `4670fe3b25a240d3aa735c0d1ff1b249`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:55.108724 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e6a1c6c1-2b1f-443e-b40a-14f345d0cdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:55.178432 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1294/2595] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:07:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:07:55.189778 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a70f5-d88e-4a24-b27f-62d6cb0fe18a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:55.191476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c94a70f5-d88e-4a24-b27f-62d6cb0fe18a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:55.248151 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a70f5-d88e-4a24-b27f-62d6cb0fe18a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:55.251758 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9127216-4ca8-4bf5-8f4b-7460840b689c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:55.252455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9127216-4ca8-4bf5-8f4b-7460840b689c service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:55.252776 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9127216-4ca8-4bf5-8f4b-7460840b689c service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:55.253046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9127216-4ca8-4bf5-8f4b-7460840b689c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:55.262536 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9127216-4ca8-4bf5-8f4b-7460840b689c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:55.263978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9127216-4ca8-4bf5-8f4b-7460840b689c service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:55.273929 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1302/2596] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:07:55.355230 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1c6c1-2b1f-443e-b40a-14f345d0cdfd None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:55.365284 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1c6c1-2b1f-443e-b40a-14f345d0cdfd None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:55.369016 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302a38d-f590-4126-850b-12f4f79c5abd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:55.369625 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302a38d-f590-4126-850b-12f4f79c5abd service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:55.369904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302a38d-f590-4126-850b-12f4f79c5abd service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:55.370172 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302a38d-f590-4126-850b-12f4f79c5abd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:55.430520 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302a38d-f590-4126-850b-12f4f79c5abd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:55.431974 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302a38d-f590-4126-850b-12f4f79c5abd service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:55.440867 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1295/2597] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:07:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:00.183806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9127216-4ca8-4bf5-8f4b-7460840b689c service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:00.231979 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:00.247570 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9127216-4ca8-4bf5-8f4b-7460840b689c service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:00.252230 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3aab91-d358-4680-869e-f20c262a8821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:00.252868 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3aab91-d358-4680-869e-f20c262a8821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:00.253146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3aab91-d358-4680-869e-f20c262a8821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:00.253426 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3aab91-d358-4680-869e-f20c262a8821 admin admin] PATH_INFO: `/v3/users/54520805f705410983bfe514b73181e0` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:00.254547 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3aab91-d358-4680-869e-f20c262a8821 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54520805f705410983bfe514b73181e0)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:00.256755 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3aab91-d358-4680-869e-f20c262a8821 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:00.304227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0e3aab91-d358-4680-869e-f20c262a8821 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54520805f705410983bfe514b73181e0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:00.304801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0e3aab91-d358-4680-869e-f20c262a8821 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54520805f705410983bfe514b73181e0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:00.312025 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0e3aab91-d358-4680-869e-f20c262a8821 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54520805f705410983bfe514b73181e0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:00.319703 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0e3aab91-d358-4680-869e-f20c262a8821 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54520805f705410983bfe514b73181e0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:00.331639 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0e3aab91-d358-4680-869e-f20c262a8821 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54520805f705410983bfe514b73181e0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:00.338114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1303/2598] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:00 2022] DELETE /identity/v3/users/54520805f705410983bfe514b73181e0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:00.342606 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918d187-a473-42d8-81c5-c69c83a0fc81 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:00.343106 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918d187-a473-42d8-81c5-c69c83a0fc81 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:00.343213 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918d187-a473-42d8-81c5-c69c83a0fc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:00.362320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d918d187-a473-42d8-81c5-c69c83a0fc81 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:00.363528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d918d187-a473-42d8-81c5-c69c83a0fc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:00.379782 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:00.428743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1296/2599] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:00.438643 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3aab91-d358-4680-869e-f20c262a8821 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:00.440064 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0e3aab91-d358-4680-869e-f20c262a8821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:00.502450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3aab91-d358-4680-869e-f20c262a8821 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:00.506395 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76033d9d-cb8b-484a-bb6c-9d2029a41cd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:00.507006 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76033d9d-cb8b-484a-bb6c-9d2029a41cd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:00.507283 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76033d9d-cb8b-484a-bb6c-9d2029a41cd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:00.507551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76033d9d-cb8b-484a-bb6c-9d2029a41cd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:00.516675 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76033d9d-cb8b-484a-bb6c-9d2029a41cd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:00.518098 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76033d9d-cb8b-484a-bb6c-9d2029a41cd8 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:00.528300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1304/2600] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:00.796445 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918d187-a473-42d8-81c5-c69c83a0fc81 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:00.797932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d918d187-a473-42d8-81c5-c69c83a0fc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:00.811384 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918d187-a473-42d8-81c5-c69c83a0fc81 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:00.815948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b18f0d-cfa5-430f-97ba-ed7442b2b6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:00.816625 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b18f0d-cfa5-430f-97ba-ed7442b2b6cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:00.816903 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b18f0d-cfa5-430f-97ba-ed7442b2b6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:00.817174 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b18f0d-cfa5-430f-97ba-ed7442b2b6cc admin admin] PATH_INFO: `/v3/projects/28fa44465115400d853799bb7589152b` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:00.819160 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b18f0d-cfa5-430f-97ba-ed7442b2b6cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=28fa44465115400d853799bb7589152b)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:00.821175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b18f0d-cfa5-430f-97ba-ed7442b2b6cc admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:00.871702 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-62b18f0d-cfa5-430f-97ba-ed7442b2b6cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28fa44465115400d853799bb7589152b', 'request_id': 'req-62b18f0d-cfa5-430f-97ba-ed7442b2b6cc'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:00.881493 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-62b18f0d-cfa5-430f-97ba-ed7442b2b6cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28fa44465115400d853799bb7589152b', 'request_id': 'req-62b18f0d-cfa5-430f-97ba-ed7442b2b6cc'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:00.887558 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-62b18f0d-cfa5-430f-97ba-ed7442b2b6cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28fa44465115400d853799bb7589152b', 'request_id': 'req-62b18f0d-cfa5-430f-97ba-ed7442b2b6cc'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:00.898981 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-62b18f0d-cfa5-430f-97ba-ed7442b2b6cc admin admin] The token cache is being invalidate because project 28fa44465115400d853799bb7589152b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:00.899382 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-62b18f0d-cfa5-430f-97ba-ed7442b2b6cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:00.900062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-62b18f0d-cfa5-430f-97ba-ed7442b2b6cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:00.901826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1297/2601] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:00 2022] DELETE /identity/v3/projects/28fa44465115400d853799bb7589152b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:00.905776 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76033d9d-cb8b-484a-bb6c-9d2029a41cd8 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:00.915900 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76033d9d-cb8b-484a-bb6c-9d2029a41cd8 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:00.979078 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af010d6b-2368-46d7-bb84-fdf20e0b612d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:00.979768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af010d6b-2368-46d7-bb84-fdf20e0b612d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:00.980051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af010d6b-2368-46d7-bb84-fdf20e0b612d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:00.980350 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af010d6b-2368-46d7-bb84-fdf20e0b612d admin admin] PATH_INFO: `/v3/users/4670fe3b25a240d3aa735c0d1ff1b249` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:00.981461 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af010d6b-2368-46d7-bb84-fdf20e0b612d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4670fe3b25a240d3aa735c0d1ff1b249)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:00.983367 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af010d6b-2368-46d7-bb84-fdf20e0b612d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:00.996610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:01.041378 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-af010d6b-2368-46d7-bb84-fdf20e0b612d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4670fe3b25a240d3aa735c0d1ff1b249'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.042046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-af010d6b-2368-46d7-bb84-fdf20e0b612d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4670fe3b25a240d3aa735c0d1ff1b249'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.051464 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-af010d6b-2368-46d7-bb84-fdf20e0b612d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4670fe3b25a240d3aa735c0d1ff1b249'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.060195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-af010d6b-2368-46d7-bb84-fdf20e0b612d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4670fe3b25a240d3aa735c0d1ff1b249'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.073955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-af010d6b-2368-46d7-bb84-fdf20e0b612d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4670fe3b25a240d3aa735c0d1ff1b249'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.080761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1305/2602] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:00 2022] DELETE /identity/v3/users/4670fe3b25a240d3aa735c0d1ff1b249 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:01.385869 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b18f0d-cfa5-430f-97ba-ed7442b2b6cc admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:01.387540 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-62b18f0d-cfa5-430f-97ba-ed7442b2b6cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:01.401900 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b18f0d-cfa5-430f-97ba-ed7442b2b6cc admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:01.470011 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077198a-c85a-44ff-8658-b07926bc3498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:01.470687 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077198a-c85a-44ff-8658-b07926bc3498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:01.470968 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077198a-c85a-44ff-8658-b07926bc3498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:01.471244 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077198a-c85a-44ff-8658-b07926bc3498 admin admin] PATH_INFO: `/v3/projects/950cdc8329004b7d83dfed87d8d92d79` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:01.472441 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077198a-c85a-44ff-8658-b07926bc3498 admin admin] RBAC: Authorizing `identity:delete_project(project_id=950cdc8329004b7d83dfed87d8d92d79)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:01.474669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077198a-c85a-44ff-8658-b07926bc3498 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:01.526198 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:01.527560 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e077198a-c85a-44ff-8658-b07926bc3498 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '950cdc8329004b7d83dfed87d8d92d79', 'request_id': 'req-e077198a-c85a-44ff-8658-b07926bc3498'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.537327 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e077198a-c85a-44ff-8658-b07926bc3498 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '950cdc8329004b7d83dfed87d8d92d79', 'request_id': 'req-e077198a-c85a-44ff-8658-b07926bc3498'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.544318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e077198a-c85a-44ff-8658-b07926bc3498 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '950cdc8329004b7d83dfed87d8d92d79', 'request_id': 'req-e077198a-c85a-44ff-8658-b07926bc3498'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.560051 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e077198a-c85a-44ff-8658-b07926bc3498 admin admin] The token cache is being invalidate because project 950cdc8329004b7d83dfed87d8d92d79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:01.561126 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e077198a-c85a-44ff-8658-b07926bc3498 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.561873 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e077198a-c85a-44ff-8658-b07926bc3498 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:01.563683 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1298/2603] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:01 2022] DELETE /identity/v3/projects/950cdc8329004b7d83dfed87d8d92d79 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:01.569307 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b03d6-708f-4554-ba22-451feecf570e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:01.569755 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b03d6-708f-4554-ba22-451feecf570e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:01.570051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b03d6-708f-4554-ba22-451feecf570e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:01.590616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b85b03d6-708f-4554-ba22-451feecf570e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:01.591902 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b85b03d6-708f-4554-ba22-451feecf570e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:01.662550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1306/2604] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:01.671422 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09e809-0c86-4c1e-aab6-4df3db263690 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:01.671732 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09e809-0c86-4c1e-aab6-4df3db263690 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:01.672006 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09e809-0c86-4c1e-aab6-4df3db263690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:01.693576 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-6a09e809-0c86-4c1e-aab6-4df3db263690 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:01.694792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6a09e809-0c86-4c1e-aab6-4df3db263690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:01.702965 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1299/2605] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:01.707352 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b03d6-708f-4554-ba22-451feecf570e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:01.717154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b03d6-708f-4554-ba22-451feecf570e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:01.721761 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ab16e-db58-4917-a969-9e6a5de54252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:01.722363 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ab16e-db58-4917-a969-9e6a5de54252 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:01.722637 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ab16e-db58-4917-a969-9e6a5de54252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:01.722903 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ab16e-db58-4917-a969-9e6a5de54252 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:01.723545 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ab16e-db58-4917-a969-9e6a5de54252 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:01.724019 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ab16e-db58-4917-a969-9e6a5de54252 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:01.725740 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ab16e-db58-4917-a969-9e6a5de54252 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:01.738612 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1307/2606] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:01.743332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09e809-0c86-4c1e-aab6-4df3db263690 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:01.754244 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09e809-0c86-4c1e-aab6-4df3db263690 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:01.759590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be8ec5-b873-4af5-9812-887b85137987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:01.760361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be8ec5-b873-4af5-9812-887b85137987 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:01.760639 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be8ec5-b873-4af5-9812-887b85137987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:01.760908 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be8ec5-b873-4af5-9812-887b85137987 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:01.761548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be8ec5-b873-4af5-9812-887b85137987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:01.763445 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be8ec5-b873-4af5-9812-887b85137987 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:01.794942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1300/2607] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:01 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:01.798959 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ab16e-db58-4917-a969-9e6a5de54252 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:01.810939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ab16e-db58-4917-a969-9e6a5de54252 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:01.890674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1e0bf-572b-4578-b433-d933ae76bbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:01.900920 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1e0bf-572b-4578-b433-d933ae76bbc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:01.901288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1e0bf-572b-4578-b433-d933ae76bbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:01.901648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1e0bf-572b-4578-b433-d933ae76bbc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:01.902611 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1e0bf-572b-4578-b433-d933ae76bbc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:01.904880 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1e0bf-572b-4578-b433-d933ae76bbc9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:01.931586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1308/2608] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:01 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:08:01.935723 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be8ec5-b873-4af5-9812-887b85137987 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:01.947369 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be8ec5-b873-4af5-9812-887b85137987 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:01.953159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a4fe2-cd97-43b4-ae0b-c7cadc84ee3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:01.955674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a4fe2-cd97-43b4-ae0b-c7cadc84ee3e admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:01.955979 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a4fe2-cd97-43b4-ae0b-c7cadc84ee3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:01.956290 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a4fe2-cd97-43b4-ae0b-c7cadc84ee3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:01.956968 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a4fe2-cd97-43b4-ae0b-c7cadc84ee3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:01.958710 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a4fe2-cd97-43b4-ae0b-c7cadc84ee3e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:01.970571 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1301/2609] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:08:01.975533 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1e0bf-572b-4578-b433-d933ae76bbc9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:01.983391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:01.988176 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1e0bf-572b-4578-b433-d933ae76bbc9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:01.995913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c954b1-8053-469f-8ecb-923412a7da2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:01.996946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c954b1-8053-469f-8ecb-923412a7da2d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:01.997578 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c954b1-8053-469f-8ecb-923412a7da2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:01.997932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c954b1-8053-469f-8ecb-923412a7da2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:01.999129 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c954b1-8053-469f-8ecb-923412a7da2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:02.002140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c954b1-8053-469f-8ecb-923412a7da2d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:02.016048 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1309/2610] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:02.020341 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a4fe2-cd97-43b4-ae0b-c7cadc84ee3e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:02.031376 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a4fe2-cd97-43b4-ae0b-c7cadc84ee3e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:02.039153 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c425d66-3bf7-4cdc-b5d5-336f0b344f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:02.039896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c425d66-3bf7-4cdc-b5d5-336f0b344f21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:02.040211 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c425d66-3bf7-4cdc-b5d5-336f0b344f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:02.040488 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c425d66-3bf7-4cdc-b5d5-336f0b344f21 admin admin] PATH_INFO: `/v3/projects/6ae11287e9794e0f967b07b9b7bd6ffb/users/a72949c143d74b7c986e53ffb57945ab/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:02.059978 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c425d66-3bf7-4cdc-b5d5-336f0b344f21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ae11287e9794e0f967b07b9b7bd6ffb, user_id=a72949c143d74b7c986e53ffb57945ab, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:02.062599 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c425d66-3bf7-4cdc-b5d5-336f0b344f21 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:02.081831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1302/2611] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:02 2022] PUT /identity/v3/projects/6ae11287e9794e0f967b07b9b7bd6ffb/users/a72949c143d74b7c986e53ffb57945ab/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:02.087367 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd2fd8-2f8a-44cf-a05d-ee240da8ed31 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:02.087649 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd2fd8-2f8a-44cf-a05d-ee240da8ed31 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:02.087980 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd2fd8-2f8a-44cf-a05d-ee240da8ed31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:02.112716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-24bd2fd8-2f8a-44cf-a05d-ee240da8ed31 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:02.113900 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-24bd2fd8-2f8a-44cf-a05d-ee240da8ed31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:02.180270 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1310/2612] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:02.190675 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c425d66-3bf7-4cdc-b5d5-336f0b344f21 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:02.193236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3c425d66-3bf7-4cdc-b5d5-336f0b344f21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:02.257306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c425d66-3bf7-4cdc-b5d5-336f0b344f21 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:02.266416 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56960316-3715-4f99-b2b5-2e2c264b59c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:02.267140 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56960316-3715-4f99-b2b5-2e2c264b59c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:02.267463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56960316-3715-4f99-b2b5-2e2c264b59c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:02.267744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56960316-3715-4f99-b2b5-2e2c264b59c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:02.279182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56960316-3715-4f99-b2b5-2e2c264b59c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:02.280385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56960316-3715-4f99-b2b5-2e2c264b59c8 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:02.304888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1303/2613] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:02.997683 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb519d-b4e5-4214-b31a-ce88f03a9faf None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:02.997683 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb519d-b4e5-4214-b31a-ce88f03a9faf None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:02.997683 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb519d-b4e5-4214-b31a-ce88f03a9faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:03.031457 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-60cb519d-b4e5-4214-b31a-ce88f03a9faf None None] MFA Rules not processed for user `a72949c143d74b7c986e53ffb57945ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:03.032994 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-60cb519d-b4e5-4214-b31a-ce88f03a9faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:03.139480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1311/2614] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:03.144347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56960316-3715-4f99-b2b5-2e2c264b59c8 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:03.164652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56960316-3715-4f99-b2b5-2e2c264b59c8 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:03.173689 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842d355-ca0b-4ae5-be0a-ca5828814201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:03.174994 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842d355-ca0b-4ae5-be0a-ca5828814201 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:03.174994 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842d355-ca0b-4ae5-be0a-ca5828814201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:03.174994 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842d355-ca0b-4ae5-be0a-ca5828814201 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:03.175616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842d355-ca0b-4ae5-be0a-ca5828814201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:03.178864 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842d355-ca0b-4ae5-be0a-ca5828814201 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:03.222537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1304/2615] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:03 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:03.228589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb519d-b4e5-4214-b31a-ce88f03a9faf None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:03.250547 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb519d-b4e5-4214-b31a-ce88f03a9faf None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:03.339658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641cfc2-2e68-4288-aa3d-9370c7c9397d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:03.340549 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641cfc2-2e68-4288-aa3d-9370c7c9397d admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:03.340818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641cfc2-2e68-4288-aa3d-9370c7c9397d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:03.341085 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641cfc2-2e68-4288-aa3d-9370c7c9397d admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:03.342461 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641cfc2-2e68-4288-aa3d-9370c7c9397d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:03.344439 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641cfc2-2e68-4288-aa3d-9370c7c9397d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:03.369184 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1312/2616] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:03 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:08:03.373065 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842d355-ca0b-4ae5-be0a-ca5828814201 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:03.386013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842d355-ca0b-4ae5-be0a-ca5828814201 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:03.391200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61498d03-9dc4-4e34-800e-e156f176d323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:03.391850 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61498d03-9dc4-4e34-800e-e156f176d323 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:03.392883 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61498d03-9dc4-4e34-800e-e156f176d323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:03.393239 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61498d03-9dc4-4e34-800e-e156f176d323 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:03.393943 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61498d03-9dc4-4e34-800e-e156f176d323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:03.396262 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61498d03-9dc4-4e34-800e-e156f176d323 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:03.412516 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1305/2617] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:03.413916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641cfc2-2e68-4288-aa3d-9370c7c9397d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:03.424335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641cfc2-2e68-4288-aa3d-9370c7c9397d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:03.429122 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6505a1c-9091-4653-85e2-15b124285856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:03.429803 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6505a1c-9091-4653-85e2-15b124285856 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:03.430080 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6505a1c-9091-4653-85e2-15b124285856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:03.430349 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6505a1c-9091-4653-85e2-15b124285856 admin admin] PATH_INFO: `/v3/projects/7f951df5d72a44c19ba472eaf6278665/users/a48ce95e8c754853865fa982a3b2b97d/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:03.445295 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:03.453282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6505a1c-9091-4653-85e2-15b124285856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f951df5d72a44c19ba472eaf6278665, user_id=a48ce95e8c754853865fa982a3b2b97d, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:03.455939 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6505a1c-9091-4653-85e2-15b124285856 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:03.472810 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1313/2618] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:03 2022] PUT /identity/v3/projects/7f951df5d72a44c19ba472eaf6278665/users/a48ce95e8c754853865fa982a3b2b97d/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:03.479433 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61498d03-9dc4-4e34-800e-e156f176d323 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:03.492912 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61498d03-9dc4-4e34-800e-e156f176d323 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:03.546380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:03.566521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f271156-dfb9-4805-8b79-5cdc421135df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:03.567734 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f271156-dfb9-4805-8b79-5cdc421135df admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:03.568044 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f271156-dfb9-4805-8b79-5cdc421135df admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:03.568381 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f271156-dfb9-4805-8b79-5cdc421135df admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:03.569044 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f271156-dfb9-4805-8b79-5cdc421135df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:03.570787 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f271156-dfb9-4805-8b79-5cdc421135df admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:03.587007 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1306/2619] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:03.588926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6505a1c-9091-4653-85e2-15b124285856 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:03.600701 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6505a1c-9091-4653-85e2-15b124285856 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:03.606060 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231491b6-aa33-414b-bcd5-c7ffea293368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:03.606732 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231491b6-aa33-414b-bcd5-c7ffea293368 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:03.607022 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231491b6-aa33-414b-bcd5-c7ffea293368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:03.607294 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231491b6-aa33-414b-bcd5-c7ffea293368 admin admin] PATH_INFO: `/v3/projects/7f951df5d72a44c19ba472eaf6278665/users/a48ce95e8c754853865fa982a3b2b97d/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:03.609940 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231491b6-aa33-414b-bcd5-c7ffea293368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f951df5d72a44c19ba472eaf6278665, user_id=a48ce95e8c754853865fa982a3b2b97d, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:03.612834 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231491b6-aa33-414b-bcd5-c7ffea293368 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:03.627157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1314/2620] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:03 2022] PUT /identity/v3/projects/7f951df5d72a44c19ba472eaf6278665/users/a48ce95e8c754853865fa982a3b2b97d/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:04.594846 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ff877-b92c-4fd1-bc64-433b9a51bda1 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.595863 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ff877-b92c-4fd1-bc64-433b9a51bda1 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.596648 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ff877-b92c-4fd1-bc64-433b9a51bda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.630564 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-653ff877-b92c-4fd1-bc64-433b9a51bda1 None None] MFA Rules not processed for user `a48ce95e8c754853865fa982a3b2b97d`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:04.631934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-653ff877-b92c-4fd1-bc64-433b9a51bda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:04.705358 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1307/2621] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:04.721541 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231491b6-aa33-414b-bcd5-c7ffea293368 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.724781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-231491b6-aa33-414b-bcd5-c7ffea293368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:04.788403 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231491b6-aa33-414b-bcd5-c7ffea293368 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.792271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd3fd3-9f91-493c-8741-c1e2ec72b9e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.792906 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd3fd3-9f91-493c-8741-c1e2ec72b9e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.793204 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd3fd3-9f91-493c-8741-c1e2ec72b9e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.793462 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd3fd3-9f91-493c-8741-c1e2ec72b9e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.859627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd3fd3-9f91-493c-8741-c1e2ec72b9e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.861326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd3fd3-9f91-493c-8741-c1e2ec72b9e0 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.881531 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1315/2622] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:06.290738 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ff877-b92c-4fd1-bc64-433b9a51bda1 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:06.304367 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ff877-b92c-4fd1-bc64-433b9a51bda1 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:06.309178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7190569-5ac4-4b33-8737-bf7028e18e89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:06.310073 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7190569-5ac4-4b33-8737-bf7028e18e89 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:06.310412 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7190569-5ac4-4b33-8737-bf7028e18e89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:06.310853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7190569-5ac4-4b33-8737-bf7028e18e89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:06.321831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7190569-5ac4-4b33-8737-bf7028e18e89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:06.323621 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7190569-5ac4-4b33-8737-bf7028e18e89 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:06.335223 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1308/2623] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:08.266476 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd3fd3-9f91-493c-8741-c1e2ec72b9e0 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:08.320811 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd3fd3-9f91-493c-8741-c1e2ec72b9e0 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:08.324370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733360f-b769-4c26-990d-20e9da32fb31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:08.324970 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733360f-b769-4c26-990d-20e9da32fb31 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.325247 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733360f-b769-4c26-990d-20e9da32fb31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.325517 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733360f-b769-4c26-990d-20e9da32fb31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.327256 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2733360f-b769-4c26-990d-20e9da32fb31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:08.371313 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:08.389865 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733360f-b769-4c26-990d-20e9da32fb31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:08.391286 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733360f-b769-4c26-990d-20e9da32fb31 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:08.401558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1316/2624] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:09.982494 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7190569-5ac4-4b33-8737-bf7028e18e89 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:09.984796 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e7190569-5ac4-4b33-8737-bf7028e18e89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:09.997606 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7190569-5ac4-4b33-8737-bf7028e18e89 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:10.001182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f8aa4-3b27-4864-83ce-66ce5610aee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:10.002029 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f8aa4-3b27-4864-83ce-66ce5610aee2 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:10.002312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f8aa4-3b27-4864-83ce-66ce5610aee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:10.002581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f8aa4-3b27-4864-83ce-66ce5610aee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:10.005437 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-801f8aa4-3b27-4864-83ce-66ce5610aee2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:10.018262 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f8aa4-3b27-4864-83ce-66ce5610aee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:10.021221 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f8aa4-3b27-4864-83ce-66ce5610aee2 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:10.032312 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1309/2625] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:12.191854 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733360f-b769-4c26-990d-20e9da32fb31 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:12.195883 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2733360f-b769-4c26-990d-20e9da32fb31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:12.233599 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733360f-b769-4c26-990d-20e9da32fb31 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:12.239191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f897e9-7355-40a9-8063-65fc029a7b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:12.240388 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f897e9-7355-40a9-8063-65fc029a7b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:12.240858 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f897e9-7355-40a9-8063-65fc029a7b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:12.241319 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f897e9-7355-40a9-8063-65fc029a7b09 admin admin] PATH_INFO: `/v3/users/bd0929f371e4442eb4eb6e12a96bbfc4` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:12.242669 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f897e9-7355-40a9-8063-65fc029a7b09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd0929f371e4442eb4eb6e12a96bbfc4)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:12.244946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f897e9-7355-40a9-8063-65fc029a7b09 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:12.308972 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f7f897e9-7355-40a9-8063-65fc029a7b09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd0929f371e4442eb4eb6e12a96bbfc4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:12.309627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f7f897e9-7355-40a9-8063-65fc029a7b09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd0929f371e4442eb4eb6e12a96bbfc4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:12.319626 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f7f897e9-7355-40a9-8063-65fc029a7b09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd0929f371e4442eb4eb6e12a96bbfc4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:12.328917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f7f897e9-7355-40a9-8063-65fc029a7b09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd0929f371e4442eb4eb6e12a96bbfc4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:12.349213 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f7f897e9-7355-40a9-8063-65fc029a7b09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd0929f371e4442eb4eb6e12a96bbfc4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:12.357460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1317/2626] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:12 2022] DELETE /identity/v3/users/bd0929f371e4442eb4eb6e12a96bbfc4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:12.674711 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f8aa4-3b27-4864-83ce-66ce5610aee2 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:12.685431 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f8aa4-3b27-4864-83ce-66ce5610aee2 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:12.770764 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6b303-86cf-49fe-817a-676c45d13221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:12.771415 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6b303-86cf-49fe-817a-676c45d13221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:12.771698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6b303-86cf-49fe-817a-676c45d13221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:12.771990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6b303-86cf-49fe-817a-676c45d13221 admin admin] PATH_INFO: `/v3/projects/7e6183eeb3ff43e79eccb5d7ad08c53c` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:12.773180 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd6b303-86cf-49fe-817a-676c45d13221 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e6183eeb3ff43e79eccb5d7ad08c53c)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:12.775132 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd6b303-86cf-49fe-817a-676c45d13221 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:12.824442 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:12.835971 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6bd6b303-86cf-49fe-817a-676c45d13221 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e6183eeb3ff43e79eccb5d7ad08c53c', 'request_id': 'req-6bd6b303-86cf-49fe-817a-676c45d13221'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:12.848689 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6bd6b303-86cf-49fe-817a-676c45d13221 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e6183eeb3ff43e79eccb5d7ad08c53c', 'request_id': 'req-6bd6b303-86cf-49fe-817a-676c45d13221'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:12.855423 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6bd6b303-86cf-49fe-817a-676c45d13221 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e6183eeb3ff43e79eccb5d7ad08c53c', 'request_id': 'req-6bd6b303-86cf-49fe-817a-676c45d13221'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:12.868055 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6bd6b303-86cf-49fe-817a-676c45d13221 admin admin] The token cache is being invalidate because project 7e6183eeb3ff43e79eccb5d7ad08c53c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:12.868376 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6bd6b303-86cf-49fe-817a-676c45d13221 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:12.869043 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6bd6b303-86cf-49fe-817a-676c45d13221 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:12.870571 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1310/2627] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:12 2022] DELETE /identity/v3/projects/7e6183eeb3ff43e79eccb5d7ad08c53c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:12.875500 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ea2f3-ae3a-4574-a51d-cfa92beac3d6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:12.875792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ea2f3-ae3a-4574-a51d-cfa92beac3d6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:12.876065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ea2f3-ae3a-4574-a51d-cfa92beac3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:12.898964 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b56ea2f3-ae3a-4574-a51d-cfa92beac3d6 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:12.900374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b56ea2f3-ae3a-4574-a51d-cfa92beac3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:13.000243 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1318/2628] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:13.005295 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e4334-c394-4971-88c6-b6613758584b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:13.005588 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e4334-c394-4971-88c6-b6613758584b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:13.005863 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e4334-c394-4971-88c6-b6613758584b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:13.026299 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-270e4334-c394-4971-88c6-b6613758584b None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:13.027497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-270e4334-c394-4971-88c6-b6613758584b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:13.036682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1311/2629] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:13.040936 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ea2f3-ae3a-4574-a51d-cfa92beac3d6 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:13.050581 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ea2f3-ae3a-4574-a51d-cfa92beac3d6 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:13.056162 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c9a09-82ac-42c5-a72d-1b0a1e296ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:13.056785 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c9a09-82ac-42c5-a72d-1b0a1e296ebd admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:13.057096 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c9a09-82ac-42c5-a72d-1b0a1e296ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:13.057341 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c9a09-82ac-42c5-a72d-1b0a1e296ebd admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:13.058002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c9a09-82ac-42c5-a72d-1b0a1e296ebd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:13.058302 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c9a09-82ac-42c5-a72d-1b0a1e296ebd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:13.059901 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c9a09-82ac-42c5-a72d-1b0a1e296ebd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:13.076318 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1319/2630] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:13.079004 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e4334-c394-4971-88c6-b6613758584b None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:13.090819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e4334-c394-4971-88c6-b6613758584b None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:13.095911 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ffd2a-7bd4-419e-9795-dd7b9b792d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:13.096765 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ffd2a-7bd4-419e-9795-dd7b9b792d24 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:13.097136 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ffd2a-7bd4-419e-9795-dd7b9b792d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:13.097604 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ffd2a-7bd4-419e-9795-dd7b9b792d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:13.098298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ffd2a-7bd4-419e-9795-dd7b9b792d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:13.100686 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ffd2a-7bd4-419e-9795-dd7b9b792d24 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:13.134499 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1312/2631] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:13 2022] POST /identity/v3/projects => generated 403 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:13.139121 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c9a09-82ac-42c5-a72d-1b0a1e296ebd admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:13.150834 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c9a09-82ac-42c5-a72d-1b0a1e296ebd admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:13.193377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:13.214364 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7a1cf-fded-43a0-8b18-3aae20d23ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:13.215107 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7a1cf-fded-43a0-8b18-3aae20d23ced admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:13.215424 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7a1cf-fded-43a0-8b18-3aae20d23ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:13.215734 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7a1cf-fded-43a0-8b18-3aae20d23ced admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:13.216689 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7a1cf-fded-43a0-8b18-3aae20d23ced admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:13.219645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7a1cf-fded-43a0-8b18-3aae20d23ced admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:13.252367 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1320/2632] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:13 2022] POST /identity/v3/users => generated 302 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:13.253708 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ffd2a-7bd4-419e-9795-dd7b9b792d24 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:13.266614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ffd2a-7bd4-419e-9795-dd7b9b792d24 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:13.288313 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3e311-2d83-4378-a40e-ef8f8bd49c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:13.288313 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3e311-2d83-4378-a40e-ef8f8bd49c29 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:13.288313 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3e311-2d83-4378-a40e-ef8f8bd49c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:13.288313 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3e311-2d83-4378-a40e-ef8f8bd49c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:13.288819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3e311-2d83-4378-a40e-ef8f8bd49c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:13.288819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3e311-2d83-4378-a40e-ef8f8bd49c29 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:13.310183 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1313/2633] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:13.311937 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7a1cf-fded-43a0-8b18-3aae20d23ced admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:13.328266 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7a1cf-fded-43a0-8b18-3aae20d23ced admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:13.333277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c41a7-e5a7-4445-b670-f4f7b6136fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:13.333932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c41a7-e5a7-4445-b670-f4f7b6136fcd admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:13.334238 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c41a7-e5a7-4445-b670-f4f7b6136fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:13.334546 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c41a7-e5a7-4445-b670-f4f7b6136fcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:13.335242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c41a7-e5a7-4445-b670-f4f7b6136fcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:13.336918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c41a7-e5a7-4445-b670-f4f7b6136fcd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:13.354133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1321/2634] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:08:13.355634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3e311-2d83-4378-a40e-ef8f8bd49c29 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:13.372820 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3e311-2d83-4378-a40e-ef8f8bd49c29 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:13.379311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b2cdaf-0919-44ee-b957-a8f4d272660d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:13.380177 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2cdaf-0919-44ee-b957-a8f4d272660d admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:13.380538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2cdaf-0919-44ee-b957-a8f4d272660d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:13.380929 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2cdaf-0919-44ee-b957-a8f4d272660d admin admin] PATH_INFO: `/v3/projects/3a14cc7ebe814b1994f69346adcd2b6b/users/17e912fc065748eea3544a2c692fcb13/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:13.399860 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b2cdaf-0919-44ee-b957-a8f4d272660d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a14cc7ebe814b1994f69346adcd2b6b, user_id=17e912fc065748eea3544a2c692fcb13, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:13.402640 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b2cdaf-0919-44ee-b957-a8f4d272660d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:13.423814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1314/2635] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:13 2022] PUT /identity/v3/projects/3a14cc7ebe814b1994f69346adcd2b6b/users/17e912fc065748eea3544a2c692fcb13/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:13.429933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20a8e3-f5ff-4fe8-b568-e6ac04770542 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:13.430218 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20a8e3-f5ff-4fe8-b568-e6ac04770542 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:13.430491 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20a8e3-f5ff-4fe8-b568-e6ac04770542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:13.449147 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0a20a8e3-f5ff-4fe8-b568-e6ac04770542 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:13.450350 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0a20a8e3-f5ff-4fe8-b568-e6ac04770542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:13.513873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1322/2636] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:08:13.520948 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b2cdaf-0919-44ee-b957-a8f4d272660d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:13.522700 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a3b2cdaf-0919-44ee-b957-a8f4d272660d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:13.594625 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b2cdaf-0919-44ee-b957-a8f4d272660d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:13.602639 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4507c41-8b60-47a7-9b3a-971c449fd2ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:13.603369 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4507c41-8b60-47a7-9b3a-971c449fd2ad service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:13.603723 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4507c41-8b60-47a7-9b3a-971c449fd2ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:13.604071 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4507c41-8b60-47a7-9b3a-971c449fd2ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:13.614733 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4507c41-8b60-47a7-9b3a-971c449fd2ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:13.616681 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4507c41-8b60-47a7-9b3a-971c449fd2ad service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:13.628374 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1315/2637] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:14.620031 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20a8e3-f5ff-4fe8-b568-e6ac04770542 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:14.621573 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0a20a8e3-f5ff-4fe8-b568-e6ac04770542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:14.638916 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20a8e3-f5ff-4fe8-b568-e6ac04770542 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:14.644425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49eb32-443d-4b14-9f78-207095dde097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:14.645149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49eb32-443d-4b14-9f78-207095dde097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:14.645504 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49eb32-443d-4b14-9f78-207095dde097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:14.645909 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49eb32-443d-4b14-9f78-207095dde097 admin admin] PATH_INFO: `/v3/users/f0d0eb9dc0f743c3985218901334b336` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:14.648014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49eb32-443d-4b14-9f78-207095dde097 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0d0eb9dc0f743c3985218901334b336)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:14.650161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49eb32-443d-4b14-9f78-207095dde097 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:14.715415 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2b49eb32-443d-4b14-9f78-207095dde097 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0d0eb9dc0f743c3985218901334b336'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:14.716526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2b49eb32-443d-4b14-9f78-207095dde097 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0d0eb9dc0f743c3985218901334b336'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:14.727794 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2b49eb32-443d-4b14-9f78-207095dde097 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0d0eb9dc0f743c3985218901334b336'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:14.743434 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2b49eb32-443d-4b14-9f78-207095dde097 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0d0eb9dc0f743c3985218901334b336'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:14.757983 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2b49eb32-443d-4b14-9f78-207095dde097 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0d0eb9dc0f743c3985218901334b336'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:14.765191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1323/2638] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:14 2022] DELETE /identity/v3/users/f0d0eb9dc0f743c3985218901334b336 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:14.812802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cd8f3-a8ef-49e9-a906-aeaeec7f6354 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:14.813337 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cd8f3-a8ef-49e9-a906-aeaeec7f6354 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:14.813428 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cd8f3-a8ef-49e9-a906-aeaeec7f6354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:14.853075 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-6f0cd8f3-a8ef-49e9-a906-aeaeec7f6354 None None] MFA Rules not processed for user `17e912fc065748eea3544a2c692fcb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:14.854941 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6f0cd8f3-a8ef-49e9-a906-aeaeec7f6354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:14.952920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:14.968452 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1316/2639] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:14 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:14.970130 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49eb32-443d-4b14-9f78-207095dde097 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:14.972695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2b49eb32-443d-4b14-9f78-207095dde097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:14.979321 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0cd8f3-a8ef-49e9-a906-aeaeec7f6354 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:14.983128 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6f0cd8f3-a8ef-49e9-a906-aeaeec7f6354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:15.014540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:15.054665 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49eb32-443d-4b14-9f78-207095dde097 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:15.065697 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29e760-47f4-4720-a7e0-097ace0dac5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:15.066457 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29e760-47f4-4720-a7e0-097ace0dac5e admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.066803 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29e760-47f4-4720-a7e0-097ace0dac5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.067176 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29e760-47f4-4720-a7e0-097ace0dac5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.067879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa29e760-47f4-4720-a7e0-097ace0dac5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:15.070505 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa29e760-47f4-4720-a7e0-097ace0dac5e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:15.078236 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0cd8f3-a8ef-49e9-a906-aeaeec7f6354 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:15.085808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3e5ab-a7f8-4b05-853e-55c04085aaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:15.086660 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3e5ab-a7f8-4b05-853e-55c04085aaeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.087206 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3e5ab-a7f8-4b05-853e-55c04085aaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.087458 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3e5ab-a7f8-4b05-853e-55c04085aaeb admin admin] PATH_INFO: `/v3/projects/edeca01476604a70bed160e5aa97607c` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.088894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3e5ab-a7f8-4b05-853e-55c04085aaeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=edeca01476604a70bed160e5aa97607c)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:15.091637 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3e5ab-a7f8-4b05-853e-55c04085aaeb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:15.108750 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1324/2640] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:14 2022] POST /identity/v3/projects => generated 403 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:15.109823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29e760-47f4-4720-a7e0-097ace0dac5e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:15.148609 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-eaa3e5ab-a7f8-4b05-853e-55c04085aaeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edeca01476604a70bed160e5aa97607c', 'request_id': 'req-eaa3e5ab-a7f8-4b05-853e-55c04085aaeb'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:15.156672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-eaa3e5ab-a7f8-4b05-853e-55c04085aaeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edeca01476604a70bed160e5aa97607c', 'request_id': 'req-eaa3e5ab-a7f8-4b05-853e-55c04085aaeb'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:15.163661 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-eaa3e5ab-a7f8-4b05-853e-55c04085aaeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edeca01476604a70bed160e5aa97607c', 'request_id': 'req-eaa3e5ab-a7f8-4b05-853e-55c04085aaeb'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:15.177290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29e760-47f4-4720-a7e0-097ace0dac5e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:15.178587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-eaa3e5ab-a7f8-4b05-853e-55c04085aaeb admin admin] The token cache is being invalidate because project edeca01476604a70bed160e5aa97607c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:15.178883 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-eaa3e5ab-a7f8-4b05-853e-55c04085aaeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:15.179543 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-eaa3e5ab-a7f8-4b05-853e-55c04085aaeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:15.181100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1317/2641] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:14 2022] DELETE /identity/v3/projects/edeca01476604a70bed160e5aa97607c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:15.185953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade548e-22bc-400f-b41e-06cf6c90d779 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.186243 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade548e-22bc-400f-b41e-06cf6c90d779 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.186523 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade548e-22bc-400f-b41e-06cf6c90d779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.210448 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-cade548e-22bc-400f-b41e-06cf6c90d779 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:15.211668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-cade548e-22bc-400f-b41e-06cf6c90d779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:15.251699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee027549-fd26-4f5e-b70b-5dfa242a232a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:15.252355 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee027549-fd26-4f5e-b70b-5dfa242a232a admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.252658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee027549-fd26-4f5e-b70b-5dfa242a232a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.252931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee027549-fd26-4f5e-b70b-5dfa242a232a admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.253584 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee027549-fd26-4f5e-b70b-5dfa242a232a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:15.255570 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee027549-fd26-4f5e-b70b-5dfa242a232a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:15.275667 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1318/2642] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:08:15.277319 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1325/2643] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:15 2022] POST /identity/v3/users => generated 308 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:08:15.281117 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade548e-22bc-400f-b41e-06cf6c90d779 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:15.282608 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-cade548e-22bc-400f-b41e-06cf6c90d779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:15.283913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53941581-c333-4af1-900b-824cfb4b9e41 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.284289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53941581-c333-4af1-900b-824cfb4b9e41 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.284582 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53941581-c333-4af1-900b-824cfb4b9e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.296044 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade548e-22bc-400f-b41e-06cf6c90d779 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:15.301005 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f55a1-af91-4327-ae86-6bdaf4657cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:15.301543 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f55a1-af91-4327-ae86-6bdaf4657cac admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.301820 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f55a1-af91-4327-ae86-6bdaf4657cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.302087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f55a1-af91-4327-ae86-6bdaf4657cac admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.302736 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f55a1-af91-4327-ae86-6bdaf4657cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:15.304321 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f55a1-af91-4327-ae86-6bdaf4657cac admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:15.304934 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-53941581-c333-4af1-900b-824cfb4b9e41 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:15.306116 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-53941581-c333-4af1-900b-824cfb4b9e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:15.314115 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1326/2644] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:08:15.318628 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1319/2645] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:15.319114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53941581-c333-4af1-900b-824cfb4b9e41 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:15.319561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f55a1-af91-4327-ae86-6bdaf4657cac admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:15.328373 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53941581-c333-4af1-900b-824cfb4b9e41 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:15.330128 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f55a1-af91-4327-ae86-6bdaf4657cac admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:15.333667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03842094-6d0e-4fed-a0ee-74f2ca52f9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:15.334262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03842094-6d0e-4fed-a0ee-74f2ca52f9a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.334541 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03842094-6d0e-4fed-a0ee-74f2ca52f9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.334812 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03842094-6d0e-4fed-a0ee-74f2ca52f9a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.335454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03842094-6d0e-4fed-a0ee-74f2ca52f9a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:15.335588 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece991e-c674-4aab-9fdf-de0ef6067b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:15.335755 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03842094-6d0e-4fed-a0ee-74f2ca52f9a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:15.336235 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece991e-c674-4aab-9fdf-de0ef6067b73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.336515 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece991e-c674-4aab-9fdf-de0ef6067b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.336787 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece991e-c674-4aab-9fdf-de0ef6067b73 admin admin] PATH_INFO: `/v3/projects/159914aa680e45aa900a550665d13e89/users/1fa270b7f95c4862a8e1924848821ab4/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.337367 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03842094-6d0e-4fed-a0ee-74f2ca52f9a5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:15.348990 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:15.350359 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1327/2646] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:15.354356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece991e-c674-4aab-9fdf-de0ef6067b73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=159914aa680e45aa900a550665d13e89, user_id=1fa270b7f95c4862a8e1924848821ab4, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:15.354756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03842094-6d0e-4fed-a0ee-74f2ca52f9a5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:15.357917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece991e-c674-4aab-9fdf-de0ef6067b73 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:15.366190 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03842094-6d0e-4fed-a0ee-74f2ca52f9a5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:15.372888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1320/2647] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:15 2022] PUT /identity/v3/projects/159914aa680e45aa900a550665d13e89/users/1fa270b7f95c4862a8e1924848821ab4/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:15.377283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece991e-c674-4aab-9fdf-de0ef6067b73 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:15.398429 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:15.427723 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389847e-94b1-40fa-aebe-2d5eb0897ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:15.428508 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389847e-94b1-40fa-aebe-2d5eb0897ebe admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.428911 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389847e-94b1-40fa-aebe-2d5eb0897ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.429279 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389847e-94b1-40fa-aebe-2d5eb0897ebe admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.430001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389847e-94b1-40fa-aebe-2d5eb0897ebe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:15.431981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389847e-94b1-40fa-aebe-2d5eb0897ebe admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:15.440518 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece991e-c674-4aab-9fdf-de0ef6067b73 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:15.445257 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba1e79-319f-43cb-a628-04f7c62aad99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:15.445868 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba1e79-319f-43cb-a628-04f7c62aad99 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.446150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba1e79-319f-43cb-a628-04f7c62aad99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.446423 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba1e79-319f-43cb-a628-04f7c62aad99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.447079 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba1e79-319f-43cb-a628-04f7c62aad99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:15.448693 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba1e79-319f-43cb-a628-04f7c62aad99 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:15.460668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1321/2648] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:08:15.464919 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba1e79-319f-43cb-a628-04f7c62aad99 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:15.465055 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1328/2649] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:15 2022] POST /identity/v3/projects => generated 401 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:15.468539 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389847e-94b1-40fa-aebe-2d5eb0897ebe admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:15.475172 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba1e79-319f-43cb-a628-04f7c62aad99 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:15.529537 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389847e-94b1-40fa-aebe-2d5eb0897ebe admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:15.531433 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1811f-a978-49fc-9588-64f41ce24111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:15.532110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1811f-a978-49fc-9588-64f41ce24111 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.532410 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1811f-a978-49fc-9588-64f41ce24111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.532688 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1811f-a978-49fc-9588-64f41ce24111 admin admin] PATH_INFO: `/v3/projects/159914aa680e45aa900a550665d13e89/users/1fa270b7f95c4862a8e1924848821ab4/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.534747 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ef393-5779-411c-9988-6b7b59255cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:15.534892 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1811f-a978-49fc-9588-64f41ce24111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=159914aa680e45aa900a550665d13e89, user_id=1fa270b7f95c4862a8e1924848821ab4, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:15.535347 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ef393-5779-411c-9988-6b7b59255cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.535620 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ef393-5779-411c-9988-6b7b59255cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.535894 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ef393-5779-411c-9988-6b7b59255cd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.536551 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ef393-5779-411c-9988-6b7b59255cd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:15.537522 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1811f-a978-49fc-9588-64f41ce24111 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:15.538425 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ef393-5779-411c-9988-6b7b59255cd1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:15.548173 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1322/2650] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:15 2022] PUT /identity/v3/projects/159914aa680e45aa900a550665d13e89/users/1fa270b7f95c4862a8e1924848821ab4/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:15.562911 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1329/2651] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:15 2022] POST /identity/v3/users => generated 301 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:08:15.564699 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1811f-a978-49fc-9588-64f41ce24111 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:15.631519 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1811f-a978-49fc-9588-64f41ce24111 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:15.636535 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993fd485-fe76-4b6d-9c22-5d3e1efaa7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:15.637151 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fd485-fe76-4b6d-9c22-5d3e1efaa7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.637460 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fd485-fe76-4b6d-9c22-5d3e1efaa7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.637772 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fd485-fe76-4b6d-9c22-5d3e1efaa7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.638539 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993fd485-fe76-4b6d-9c22-5d3e1efaa7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:15.640184 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993fd485-fe76-4b6d-9c22-5d3e1efaa7d0 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:15.654641 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1323/2652] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:15.655852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ef393-5779-411c-9988-6b7b59255cd1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:15.666421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ef393-5779-411c-9988-6b7b59255cd1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:15.671430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e4310-337b-44ff-ad86-bac26da5c7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:15.672038 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e4310-337b-44ff-ad86-bac26da5c7f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.672385 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e4310-337b-44ff-ad86-bac26da5c7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.672654 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e4310-337b-44ff-ad86-bac26da5c7f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.673270 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e4310-337b-44ff-ad86-bac26da5c7f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:15.675614 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e4310-337b-44ff-ad86-bac26da5c7f5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:15.686480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1330/2653] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:08:15.691218 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993fd485-fe76-4b6d-9c22-5d3e1efaa7d0 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:15.702550 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993fd485-fe76-4b6d-9c22-5d3e1efaa7d0 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:15.708713 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc70193-ff1d-4fe9-938a-dfb451edfc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:15.708986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc70193-ff1d-4fe9-938a-dfb451edfc24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.709339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc70193-ff1d-4fe9-938a-dfb451edfc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.709681 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc70193-ff1d-4fe9-938a-dfb451edfc24 admin admin] PATH_INFO: `/v3/projects/008e6e57828541a4b4f3583bc547ae21/users/2b28c820f7744cef94d26ff36df2e632/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.727396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc70193-ff1d-4fe9-938a-dfb451edfc24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=008e6e57828541a4b4f3583bc547ae21, user_id=2b28c820f7744cef94d26ff36df2e632, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:15.730053 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc70193-ff1d-4fe9-938a-dfb451edfc24 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:15.734446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:15.752066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1324/2654] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:15 2022] PUT /identity/v3/projects/008e6e57828541a4b4f3583bc547ae21/users/2b28c820f7744cef94d26ff36df2e632/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:15.763722 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2536fa5-d948-4d6d-8f7b-61f7b2febd58 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.764704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2536fa5-d948-4d6d-8f7b-61f7b2febd58 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.765042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2536fa5-d948-4d6d-8f7b-61f7b2febd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.799285 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b2536fa5-d948-4d6d-8f7b-61f7b2febd58 None None] MFA Rules not processed for user `2b28c820f7744cef94d26ff36df2e632`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:15.800858 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b2536fa5-d948-4d6d-8f7b-61f7b2febd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:15.893942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1331/2655] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:15 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:15.894777 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc70193-ff1d-4fe9-938a-dfb451edfc24 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:15.967891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc70193-ff1d-4fe9-938a-dfb451edfc24 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:15.974546 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef61526-f31e-4840-b800-1f1f4f2f7f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:15.975116 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef61526-f31e-4840-b800-1f1f4f2f7f5c admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.975403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef61526-f31e-4840-b800-1f1f4f2f7f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.975680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef61526-f31e-4840-b800-1f1f4f2f7f5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.976492 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef61526-f31e-4840-b800-1f1f4f2f7f5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:15.978531 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef61526-f31e-4840-b800-1f1f4f2f7f5c admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:16.059542 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1325/2656] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:15 2022] POST /identity/v3/projects => generated 403 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:08:16.060926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2536fa5-d948-4d6d-8f7b-61f7b2febd58 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:16.060926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2536fa5-d948-4d6d-8f7b-61f7b2febd58 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:16.107324 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:16.151754 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8e92f-1191-4183-9d17-76dcc0a0017d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:16.152380 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8e92f-1191-4183-9d17-76dcc0a0017d admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:16.152661 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8e92f-1191-4183-9d17-76dcc0a0017d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:16.152933 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8e92f-1191-4183-9d17-76dcc0a0017d admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:16.153747 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8e92f-1191-4183-9d17-76dcc0a0017d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:16.155674 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8e92f-1191-4183-9d17-76dcc0a0017d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:16.178749 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1332/2657] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:16 2022] POST /identity/v3/users => generated 308 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:16.182772 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef61526-f31e-4840-b800-1f1f4f2f7f5c admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:16.194260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef61526-f31e-4840-b800-1f1f4f2f7f5c admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:16.199268 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53574002-f372-4aaa-b77b-ce130b6eac44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:16.200331 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53574002-f372-4aaa-b77b-ce130b6eac44 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:16.200830 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53574002-f372-4aaa-b77b-ce130b6eac44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:16.201111 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53574002-f372-4aaa-b77b-ce130b6eac44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:16.201772 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53574002-f372-4aaa-b77b-ce130b6eac44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:16.203408 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53574002-f372-4aaa-b77b-ce130b6eac44 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:16.215285 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1326/2658] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:08:16.219447 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8e92f-1191-4183-9d17-76dcc0a0017d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:16.230260 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8e92f-1191-4183-9d17-76dcc0a0017d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:16.235468 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907e79e-4acc-47f1-bf35-707b75dbfa10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:16.236275 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907e79e-4acc-47f1-bf35-707b75dbfa10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:16.236628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907e79e-4acc-47f1-bf35-707b75dbfa10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:16.236993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907e79e-4acc-47f1-bf35-707b75dbfa10 admin admin] PATH_INFO: `/v3/projects/00be44a9bfd3482798a17a3e6efe53bc/users/f3bffa2156dc4a22a363c6de71a1a914/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:16.256687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907e79e-4acc-47f1-bf35-707b75dbfa10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00be44a9bfd3482798a17a3e6efe53bc, user_id=f3bffa2156dc4a22a363c6de71a1a914, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:16.259455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907e79e-4acc-47f1-bf35-707b75dbfa10 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:16.268951 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da841614-68d2-4a05-8f32-d98fe8ec41d2 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:16.269242 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da841614-68d2-4a05-8f32-d98fe8ec41d2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:16.269518 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da841614-68d2-4a05-8f32-d98fe8ec41d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:16.277863 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1333/2659] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:16 2022] PUT /identity/v3/projects/00be44a9bfd3482798a17a3e6efe53bc/users/f3bffa2156dc4a22a363c6de71a1a914/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:16.282796 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907e79e-4acc-47f1-bf35-707b75dbfa10 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:16.300348 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-da841614-68d2-4a05-8f32-d98fe8ec41d2 None None] MFA Rules not processed for user `1fa270b7f95c4862a8e1924848821ab4`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:16.301599 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-da841614-68d2-4a05-8f32-d98fe8ec41d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:16.359045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907e79e-4acc-47f1-bf35-707b75dbfa10 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:16.364980 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7d90d-0eb5-4d70-a6d4-f178a0972c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:16.365705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7d90d-0eb5-4d70-a6d4-f178a0972c49 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:16.366059 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7d90d-0eb5-4d70-a6d4-f178a0972c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:16.366410 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7d90d-0eb5-4d70-a6d4-f178a0972c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:16.367133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7d90d-0eb5-4d70-a6d4-f178a0972c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:16.368833 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7d90d-0eb5-4d70-a6d4-f178a0972c49 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:16.380622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1334/2660] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:16.384529 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7d90d-0eb5-4d70-a6d4-f178a0972c49 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:16.385402 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1327/2661] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:16 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:08:16.396893 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7d90d-0eb5-4d70-a6d4-f178a0972c49 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:16.402422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b014d-531d-4072-a0a7-b3b44bd3bde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:16.403644 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b014d-531d-4072-a0a7-b3b44bd3bde6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:16.404314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b014d-531d-4072-a0a7-b3b44bd3bde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:16.404834 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b014d-531d-4072-a0a7-b3b44bd3bde6 admin admin] PATH_INFO: `/v3/projects/00be44a9bfd3482798a17a3e6efe53bc/users/f3bffa2156dc4a22a363c6de71a1a914/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:16.407065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b014d-531d-4072-a0a7-b3b44bd3bde6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00be44a9bfd3482798a17a3e6efe53bc, user_id=f3bffa2156dc4a22a363c6de71a1a914, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:16.410032 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b014d-531d-4072-a0a7-b3b44bd3bde6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:16.422280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1335/2662] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:16 2022] PUT /identity/v3/projects/00be44a9bfd3482798a17a3e6efe53bc/users/f3bffa2156dc4a22a363c6de71a1a914/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:16.433392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dc8b8-c27d-4b0b-9f17-e3554fa539f9 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:16.433680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dc8b8-c27d-4b0b-9f17-e3554fa539f9 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:16.433958 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dc8b8-c27d-4b0b-9f17-e3554fa539f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:16.465098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-2d5dc8b8-c27d-4b0b-9f17-e3554fa539f9 None None] MFA Rules not processed for user `f3bffa2156dc4a22a363c6de71a1a914`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:16.466332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2d5dc8b8-c27d-4b0b-9f17-e3554fa539f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:16.512641 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:16.543578 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1328/2663] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:16 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:16.554902 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b014d-531d-4072-a0a7-b3b44bd3bde6 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:16.556649 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-796b014d-531d-4072-a0a7-b3b44bd3bde6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:16.619805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b014d-531d-4072-a0a7-b3b44bd3bde6 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:16.623365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab58dd7-a490-4919-bb3b-b60a1b20a84a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:16.623984 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab58dd7-a490-4919-bb3b-b60a1b20a84a service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:16.624298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab58dd7-a490-4919-bb3b-b60a1b20a84a service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:16.624568 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab58dd7-a490-4919-bb3b-b60a1b20a84a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:16.634390 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab58dd7-a490-4919-bb3b-b60a1b20a84a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:16.635823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab58dd7-a490-4919-bb3b-b60a1b20a84a service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:16.646212 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1336/2664] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:16 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:16.787449 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5dc8b8-c27d-4b0b-9f17-e3554fa539f9 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:16.852802 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5dc8b8-c27d-4b0b-9f17-e3554fa539f9 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:16.858344 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1413574-3ba9-4a9f-b7cc-c1af8fc03a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:16.859012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1413574-3ba9-4a9f-b7cc-c1af8fc03a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:16.859297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1413574-3ba9-4a9f-b7cc-c1af8fc03a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:16.859570 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1413574-3ba9-4a9f-b7cc-c1af8fc03a70 admin admin] PATH_INFO: `/v3/users/2b28c820f7744cef94d26ff36df2e632` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:16.861146 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1413574-3ba9-4a9f-b7cc-c1af8fc03a70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b28c820f7744cef94d26ff36df2e632)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:16.863092 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1413574-3ba9-4a9f-b7cc-c1af8fc03a70 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:16.913532 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d1413574-3ba9-4a9f-b7cc-c1af8fc03a70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b28c820f7744cef94d26ff36df2e632'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:16.925975 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d1413574-3ba9-4a9f-b7cc-c1af8fc03a70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b28c820f7744cef94d26ff36df2e632'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:16.942357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d1413574-3ba9-4a9f-b7cc-c1af8fc03a70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b28c820f7744cef94d26ff36df2e632'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:16.949736 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d1413574-3ba9-4a9f-b7cc-c1af8fc03a70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b28c820f7744cef94d26ff36df2e632'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:16.950804 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-d1413574-3ba9-4a9f-b7cc-c1af8fc03a70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b28c820f7744cef94d26ff36df2e632'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:16.957533 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1329/2665] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:16 2022] DELETE /identity/v3/users/2b28c820f7744cef94d26ff36df2e632 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:16.961966 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e50068-bb4e-43bd-a086-5ed8dfc06755 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:16.962445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e50068-bb4e-43bd-a086-5ed8dfc06755 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:16.962580 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e50068-bb4e-43bd-a086-5ed8dfc06755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:16.980982 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c6e50068-bb4e-43bd-a086-5ed8dfc06755 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:16.982187 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c6e50068-bb4e-43bd-a086-5ed8dfc06755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:16.993907 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:17.043961 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1337/2666] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:17.054054 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1413574-3ba9-4a9f-b7cc-c1af8fc03a70 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:17.055586 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d1413574-3ba9-4a9f-b7cc-c1af8fc03a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:17.115649 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1413574-3ba9-4a9f-b7cc-c1af8fc03a70 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:17.119939 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908daec9-cfc5-466a-9b42-be1aadb36eb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:17.120690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908daec9-cfc5-466a-9b42-be1aadb36eb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:17.121164 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908daec9-cfc5-466a-9b42-be1aadb36eb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:17.121744 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908daec9-cfc5-466a-9b42-be1aadb36eb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:17.133127 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908daec9-cfc5-466a-9b42-be1aadb36eb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:17.134735 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908daec9-cfc5-466a-9b42-be1aadb36eb1 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:17.142667 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:17.160199 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1330/2667] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:17.684766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e50068-bb4e-43bd-a086-5ed8dfc06755 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:17.684766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c6e50068-bb4e-43bd-a086-5ed8dfc06755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:17.695401 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e50068-bb4e-43bd-a086-5ed8dfc06755 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:17.700490 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46382179-187c-4494-848f-1cf74fe94e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:17.701334 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46382179-187c-4494-848f-1cf74fe94e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:17.701620 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46382179-187c-4494-848f-1cf74fe94e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:17.701897 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46382179-187c-4494-848f-1cf74fe94e96 admin admin] PATH_INFO: `/v3/projects/008e6e57828541a4b4f3583bc547ae21` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:17.703141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46382179-187c-4494-848f-1cf74fe94e96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=008e6e57828541a4b4f3583bc547ae21)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:17.705401 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46382179-187c-4494-848f-1cf74fe94e96 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:17.786598 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-46382179-187c-4494-848f-1cf74fe94e96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '008e6e57828541a4b4f3583bc547ae21', 'request_id': 'req-46382179-187c-4494-848f-1cf74fe94e96'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:17.793795 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-46382179-187c-4494-848f-1cf74fe94e96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '008e6e57828541a4b4f3583bc547ae21', 'request_id': 'req-46382179-187c-4494-848f-1cf74fe94e96'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:17.810977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-46382179-187c-4494-848f-1cf74fe94e96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '008e6e57828541a4b4f3583bc547ae21', 'request_id': 'req-46382179-187c-4494-848f-1cf74fe94e96'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:17.825046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-46382179-187c-4494-848f-1cf74fe94e96 admin admin] The token cache is being invalidate because project 008e6e57828541a4b4f3583bc547ae21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:17.825553 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-46382179-187c-4494-848f-1cf74fe94e96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:17.826256 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-46382179-187c-4494-848f-1cf74fe94e96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:17.829742 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1338/2668] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:17 2022] DELETE /identity/v3/projects/008e6e57828541a4b4f3583bc547ae21 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:17.835202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908daec9-cfc5-466a-9b42-be1aadb36eb1 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:17.848285 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908daec9-cfc5-466a-9b42-be1aadb36eb1 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:17.920465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40434de6-b09d-45d1-98ad-60cfcff29637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:17.921325 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40434de6-b09d-45d1-98ad-60cfcff29637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:17.921444 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40434de6-b09d-45d1-98ad-60cfcff29637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:17.921715 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40434de6-b09d-45d1-98ad-60cfcff29637 admin admin] PATH_INFO: `/v3/users/f3bffa2156dc4a22a363c6de71a1a914` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:17.922831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40434de6-b09d-45d1-98ad-60cfcff29637 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3bffa2156dc4a22a363c6de71a1a914)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:17.925775 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40434de6-b09d-45d1-98ad-60cfcff29637 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:17.981674 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-40434de6-b09d-45d1-98ad-60cfcff29637 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3bffa2156dc4a22a363c6de71a1a914'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:17.990324 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-40434de6-b09d-45d1-98ad-60cfcff29637 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3bffa2156dc4a22a363c6de71a1a914'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.009089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-40434de6-b09d-45d1-98ad-60cfcff29637 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3bffa2156dc4a22a363c6de71a1a914'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.015945 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46382179-187c-4494-848f-1cf74fe94e96 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:18.018550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-46382179-187c-4494-848f-1cf74fe94e96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:18.022683 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-40434de6-b09d-45d1-98ad-60cfcff29637 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3bffa2156dc4a22a363c6de71a1a914'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.023561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-40434de6-b09d-45d1-98ad-60cfcff29637 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3bffa2156dc4a22a363c6de71a1a914'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.030582 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1331/2669] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:17 2022] DELETE /identity/v3/users/f3bffa2156dc4a22a363c6de71a1a914 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:18.041226 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46382179-187c-4494-848f-1cf74fe94e96 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:18.102094 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca823c44-368c-4380-81bd-3d908c88e779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:18.102956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca823c44-368c-4380-81bd-3d908c88e779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.103239 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca823c44-368c-4380-81bd-3d908c88e779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.103526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca823c44-368c-4380-81bd-3d908c88e779 admin admin] PATH_INFO: `/v3/users/17e912fc065748eea3544a2c692fcb13` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.104824 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca823c44-368c-4380-81bd-3d908c88e779 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17e912fc065748eea3544a2c692fcb13)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:18.107183 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca823c44-368c-4380-81bd-3d908c88e779 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:18.152535 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:18.155107 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ca823c44-368c-4380-81bd-3d908c88e779 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17e912fc065748eea3544a2c692fcb13'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.156279 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ca823c44-368c-4380-81bd-3d908c88e779 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17e912fc065748eea3544a2c692fcb13'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.166549 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ca823c44-368c-4380-81bd-3d908c88e779 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17e912fc065748eea3544a2c692fcb13'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.173910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ca823c44-368c-4380-81bd-3d908c88e779 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17e912fc065748eea3544a2c692fcb13'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.192446 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ca823c44-368c-4380-81bd-3d908c88e779 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17e912fc065748eea3544a2c692fcb13'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.199881 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1339/2670] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:18 2022] DELETE /identity/v3/users/17e912fc065748eea3544a2c692fcb13 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:18.354839 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40434de6-b09d-45d1-98ad-60cfcff29637 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:18.356682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-40434de6-b09d-45d1-98ad-60cfcff29637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:18.370186 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40434de6-b09d-45d1-98ad-60cfcff29637 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:18.424500 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca823c44-368c-4380-81bd-3d908c88e779 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:18.426141 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ca823c44-368c-4380-81bd-3d908c88e779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:18.440040 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425907fa-6c62-4b51-a99f-cc9eb6f8c87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:18.440987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425907fa-6c62-4b51-a99f-cc9eb6f8c87a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.441268 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425907fa-6c62-4b51-a99f-cc9eb6f8c87a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.441539 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425907fa-6c62-4b51-a99f-cc9eb6f8c87a admin admin] PATH_INFO: `/v3/projects/00be44a9bfd3482798a17a3e6efe53bc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.442730 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425907fa-6c62-4b51-a99f-cc9eb6f8c87a admin admin] RBAC: Authorizing `identity:delete_project(project_id=00be44a9bfd3482798a17a3e6efe53bc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:18.444779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425907fa-6c62-4b51-a99f-cc9eb6f8c87a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:18.472496 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:18.499714 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca823c44-368c-4380-81bd-3d908c88e779 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:18.499925 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-425907fa-6c62-4b51-a99f-cc9eb6f8c87a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00be44a9bfd3482798a17a3e6efe53bc', 'request_id': 'req-425907fa-6c62-4b51-a99f-cc9eb6f8c87a'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.508200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-425907fa-6c62-4b51-a99f-cc9eb6f8c87a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00be44a9bfd3482798a17a3e6efe53bc', 'request_id': 'req-425907fa-6c62-4b51-a99f-cc9eb6f8c87a'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.513819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-425907fa-6c62-4b51-a99f-cc9eb6f8c87a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00be44a9bfd3482798a17a3e6efe53bc', 'request_id': 'req-425907fa-6c62-4b51-a99f-cc9eb6f8c87a'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.527091 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-425907fa-6c62-4b51-a99f-cc9eb6f8c87a admin admin] The token cache is being invalidate because project 00be44a9bfd3482798a17a3e6efe53bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:18.527376 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-425907fa-6c62-4b51-a99f-cc9eb6f8c87a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.528383 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-425907fa-6c62-4b51-a99f-cc9eb6f8c87a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.530888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1332/2671] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:18 2022] DELETE /identity/v3/projects/00be44a9bfd3482798a17a3e6efe53bc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:18.537073 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf7e4e-c14a-47a3-afb1-b438654a8bda None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.537364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf7e4e-c14a-47a3-afb1-b438654a8bda None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.537651 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf7e4e-c14a-47a3-afb1-b438654a8bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.558354 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f3cf7e4e-c14a-47a3-afb1-b438654a8bda None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:18.559540 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f3cf7e4e-c14a-47a3-afb1-b438654a8bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:18.567386 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1715b49-f2e4-4acd-97ed-2ca2dcf1c841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:18.568019 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1715b49-f2e4-4acd-97ed-2ca2dcf1c841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.568340 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1715b49-f2e4-4acd-97ed-2ca2dcf1c841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.568614 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1715b49-f2e4-4acd-97ed-2ca2dcf1c841 admin admin] PATH_INFO: `/v3/projects/3a14cc7ebe814b1994f69346adcd2b6b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.570571 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1715b49-f2e4-4acd-97ed-2ca2dcf1c841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a14cc7ebe814b1994f69346adcd2b6b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:18.573332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1715b49-f2e4-4acd-97ed-2ca2dcf1c841 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:18.621441 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1333/2672] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:08:18.625036 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a1715b49-f2e4-4acd-97ed-2ca2dcf1c841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a14cc7ebe814b1994f69346adcd2b6b', 'request_id': 'req-a1715b49-f2e4-4acd-97ed-2ca2dcf1c841'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.630877 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6787e-f78d-4c0c-a3a4-b02d066dad4e None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.631508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6787e-f78d-4c0c-a3a4-b02d066dad4e None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.631819 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6787e-f78d-4c0c-a3a4-b02d066dad4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.632261 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a1715b49-f2e4-4acd-97ed-2ca2dcf1c841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a14cc7ebe814b1994f69346adcd2b6b', 'request_id': 'req-a1715b49-f2e4-4acd-97ed-2ca2dcf1c841'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.640534 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a1715b49-f2e4-4acd-97ed-2ca2dcf1c841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a14cc7ebe814b1994f69346adcd2b6b', 'request_id': 'req-a1715b49-f2e4-4acd-97ed-2ca2dcf1c841'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.649393 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:18.653533 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-a4a6787e-f78d-4c0c-a3a4-b02d066dad4e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:18.653896 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a1715b49-f2e4-4acd-97ed-2ca2dcf1c841 admin admin] The token cache is being invalidate because project 3a14cc7ebe814b1994f69346adcd2b6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:18.654532 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a1715b49-f2e4-4acd-97ed-2ca2dcf1c841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.654817 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a4a6787e-f78d-4c0c-a3a4-b02d066dad4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:18.655647 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a1715b49-f2e4-4acd-97ed-2ca2dcf1c841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.658517 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1340/2673] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:18 2022] DELETE /identity/v3/projects/3a14cc7ebe814b1994f69346adcd2b6b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:18.662054 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1715b49-f2e4-4acd-97ed-2ca2dcf1c841 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:18.663477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a1715b49-f2e4-4acd-97ed-2ca2dcf1c841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:18.729688 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1334/2674] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:18.730882 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1715b49-f2e4-4acd-97ed-2ca2dcf1c841 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:18.734318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6787e-f78d-4c0c-a3a4-b02d066dad4e None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:18.735991 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54dfc1-749a-42d3-97df-9136d86e8828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:18.736658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54dfc1-749a-42d3-97df-9136d86e8828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.736938 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54dfc1-749a-42d3-97df-9136d86e8828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.737212 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54dfc1-749a-42d3-97df-9136d86e8828 admin admin] PATH_INFO: `/v3/users/1fa270b7f95c4862a8e1924848821ab4` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.742283 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54dfc1-749a-42d3-97df-9136d86e8828 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fa270b7f95c4862a8e1924848821ab4)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:18.743397 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6787e-f78d-4c0c-a3a4-b02d066dad4e None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:18.744274 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54dfc1-749a-42d3-97df-9136d86e8828 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:18.748188 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d2ec7-d6ab-4ceb-aac4-e33216d71c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:18.748785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d2ec7-d6ab-4ceb-aac4-e33216d71c69 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.749064 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d2ec7-d6ab-4ceb-aac4-e33216d71c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.749335 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d2ec7-d6ab-4ceb-aac4-e33216d71c69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.749973 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d2ec7-d6ab-4ceb-aac4-e33216d71c69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:18.750272 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d2ec7-d6ab-4ceb-aac4-e33216d71c69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:18.752202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d2ec7-d6ab-4ceb-aac4-e33216d71c69 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:18.773710 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1335/2675] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:18.778288 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d2ec7-d6ab-4ceb-aac4-e33216d71c69 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:18.789286 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d2ec7-d6ab-4ceb-aac4-e33216d71c69 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:18.793987 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb430a7-9641-4337-910f-601dce0e09e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:18.794569 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb430a7-9641-4337-910f-601dce0e09e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.794848 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb430a7-9641-4337-910f-601dce0e09e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.795114 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb430a7-9641-4337-910f-601dce0e09e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.795724 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb430a7-9641-4337-910f-601dce0e09e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:18.797410 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ee54dfc1-749a-42d3-97df-9136d86e8828 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fa270b7f95c4862a8e1924848821ab4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.797625 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb430a7-9641-4337-910f-601dce0e09e4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:18.798765 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ee54dfc1-749a-42d3-97df-9136d86e8828 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fa270b7f95c4862a8e1924848821ab4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.807684 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ee54dfc1-749a-42d3-97df-9136d86e8828 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fa270b7f95c4862a8e1924848821ab4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.815791 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ee54dfc1-749a-42d3-97df-9136d86e8828 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fa270b7f95c4862a8e1924848821ab4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.831857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1336/2676] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:18 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:08:18.832075 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ee54dfc1-749a-42d3-97df-9136d86e8828 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fa270b7f95c4862a8e1924848821ab4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:18.833091 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb430a7-9641-4337-910f-601dce0e09e4 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:18.834747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1bb430a7-9641-4337-910f-601dce0e09e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:18.835237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1341/2677] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:18 2022] DELETE /identity/v3/users/1fa270b7f95c4862a8e1924848821ab4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:18.852820 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:18.905075 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb430a7-9641-4337-910f-601dce0e09e4 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:18.973982 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c024dbd-c4b1-4c37-85d8-0fa9d7107446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:18.974654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c024dbd-c4b1-4c37-85d8-0fa9d7107446 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.974868 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c024dbd-c4b1-4c37-85d8-0fa9d7107446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.975141 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c024dbd-c4b1-4c37-85d8-0fa9d7107446 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.975772 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c024dbd-c4b1-4c37-85d8-0fa9d7107446 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:18.978065 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c024dbd-c4b1-4c37-85d8-0fa9d7107446 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.008127 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1337/2678] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:18 2022] POST /identity/v3/users => generated 294 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:19.011837 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54dfc1-749a-42d3-97df-9136d86e8828 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.023094 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54dfc1-749a-42d3-97df-9136d86e8828 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.028069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ffdda-c5d1-4294-8ba5-ce69b3eb67c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.028725 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ffdda-c5d1-4294-8ba5-ce69b3eb67c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.029008 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ffdda-c5d1-4294-8ba5-ce69b3eb67c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.029283 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ffdda-c5d1-4294-8ba5-ce69b3eb67c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.032568 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ffdda-c5d1-4294-8ba5-ce69b3eb67c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.032568 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ffdda-c5d1-4294-8ba5-ce69b3eb67c3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.033843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c024dbd-c4b1-4c37-85d8-0fa9d7107446 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.035616 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7c024dbd-c4b1-4c37-85d8-0fa9d7107446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:19.047621 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c024dbd-c4b1-4c37-85d8-0fa9d7107446 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.052288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1342/2679] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:19.056170 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ffdda-c5d1-4294-8ba5-ce69b3eb67c3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.057319 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4b78e-64ab-437c-bafa-a5061b101633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.057561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4b78e-64ab-437c-bafa-a5061b101633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.057846 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4b78e-64ab-437c-bafa-a5061b101633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.058123 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4b78e-64ab-437c-bafa-a5061b101633 admin admin] PATH_INFO: `/v3/projects/159914aa680e45aa900a550665d13e89` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.059308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db4b78e-64ab-437c-bafa-a5061b101633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=159914aa680e45aa900a550665d13e89)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.061915 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db4b78e-64ab-437c-bafa-a5061b101633 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.068780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ffdda-c5d1-4294-8ba5-ce69b3eb67c3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.076089 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8a1da-339e-4393-b119-0adaf83e4f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.077025 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8a1da-339e-4393-b119-0adaf83e4f30 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.077507 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8a1da-339e-4393-b119-0adaf83e4f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.077871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8a1da-339e-4393-b119-0adaf83e4f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.078767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8a1da-339e-4393-b119-0adaf83e4f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.081638 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8a1da-339e-4393-b119-0adaf83e4f30 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.094408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1343/2680] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:08:19.099252 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8a1da-339e-4393-b119-0adaf83e4f30 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.109530 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8a1da-339e-4393-b119-0adaf83e4f30 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.119045 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-0db4b78e-64ab-437c-bafa-a5061b101633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '159914aa680e45aa900a550665d13e89', 'request_id': 'req-0db4b78e-64ab-437c-bafa-a5061b101633'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:19.129224 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-0db4b78e-64ab-437c-bafa-a5061b101633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '159914aa680e45aa900a550665d13e89', 'request_id': 'req-0db4b78e-64ab-437c-bafa-a5061b101633'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:19.136717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-0db4b78e-64ab-437c-bafa-a5061b101633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '159914aa680e45aa900a550665d13e89', 'request_id': 'req-0db4b78e-64ab-437c-bafa-a5061b101633'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:19.152045 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-0db4b78e-64ab-437c-bafa-a5061b101633 admin admin] The token cache is being invalidate because project 159914aa680e45aa900a550665d13e89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:19.152438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-0db4b78e-64ab-437c-bafa-a5061b101633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:19.153173 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-0db4b78e-64ab-437c-bafa-a5061b101633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:19.155058 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1338/2681] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:19 2022] DELETE /identity/v3/projects/159914aa680e45aa900a550665d13e89 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:19.159605 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807cc28-0a63-4f03-a8ef-1274b858d600 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.159890 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807cc28-0a63-4f03-a8ef-1274b858d600 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.160403 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807cc28-0a63-4f03-a8ef-1274b858d600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.171271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:19.179871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919723a7-746c-4557-b3a6-95a585037b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.180739 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919723a7-746c-4557-b3a6-95a585037b57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.181575 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919723a7-746c-4557-b3a6-95a585037b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.181864 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919723a7-746c-4557-b3a6-95a585037b57 admin admin] PATH_INFO: `/v3/projects/ccd191c8d6504611a7985af2ecac5dbb/users/6ae9e8fadd7e4d57a0bc9b10e3cb0b54/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.185396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1807cc28-0a63-4f03-a8ef-1274b858d600 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:19.186617 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1807cc28-0a63-4f03-a8ef-1274b858d600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:19.197739 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1339/2682] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:08:19.201473 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919723a7-746c-4557-b3a6-95a585037b57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccd191c8d6504611a7985af2ecac5dbb, user_id=6ae9e8fadd7e4d57a0bc9b10e3cb0b54, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.205711 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919723a7-746c-4557-b3a6-95a585037b57 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.206814 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adfc2d-644b-496e-874d-84566c54e8e0 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.207230 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adfc2d-644b-496e-874d-84566c54e8e0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.207570 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adfc2d-644b-496e-874d-84566c54e8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.221609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1344/2683] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:19 2022] PUT /identity/v3/projects/ccd191c8d6504611a7985af2ecac5dbb/users/6ae9e8fadd7e4d57a0bc9b10e3cb0b54/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:19.233009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-25adfc2d-644b-496e-874d-84566c54e8e0 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:19.233846 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d0732-eb9d-42e2-b914-7165a4c4dda4 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.234143 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d0732-eb9d-42e2-b914-7165a4c4dda4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.234418 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d0732-eb9d-42e2-b914-7165a4c4dda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.234575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-25adfc2d-644b-496e-874d-84566c54e8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:19.264357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:19.268398 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c31d0732-eb9d-42e2-b914-7165a4c4dda4 None None] MFA Rules not processed for user `6ae9e8fadd7e4d57a0bc9b10e3cb0b54`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:19.269621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c31d0732-eb9d-42e2-b914-7165a4c4dda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:19.313807 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1340/2684] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:19.319428 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25adfc2d-644b-496e-874d-84566c54e8e0 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.330216 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25adfc2d-644b-496e-874d-84566c54e8e0 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.335820 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce9bdb-e1b5-4ed8-a886-630548ca6484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.336497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce9bdb-e1b5-4ed8-a886-630548ca6484 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.336773 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce9bdb-e1b5-4ed8-a886-630548ca6484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.337040 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce9bdb-e1b5-4ed8-a886-630548ca6484 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.337683 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce9bdb-e1b5-4ed8-a886-630548ca6484 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:19.337980 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce9bdb-e1b5-4ed8-a886-630548ca6484 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.339561 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce9bdb-e1b5-4ed8-a886-630548ca6484 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.348178 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1345/2685] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:19.350347 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d0732-eb9d-42e2-b914-7165a4c4dda4 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.351798 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c31d0732-eb9d-42e2-b914-7165a4c4dda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:19.352693 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1341/2686] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:19.357052 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce9bdb-e1b5-4ed8-a886-630548ca6484 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.367401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce9bdb-e1b5-4ed8-a886-630548ca6484 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.367924 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d0732-eb9d-42e2-b914-7165a4c4dda4 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.372150 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf82ea-4db5-4497-9b19-55bb9b1eb826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.372762 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6102b70-cfa9-46b1-83f6-10f9126d22d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.372917 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf82ea-4db5-4497-9b19-55bb9b1eb826 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.373131 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf82ea-4db5-4497-9b19-55bb9b1eb826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.373407 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf82ea-4db5-4497-9b19-55bb9b1eb826 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.373571 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6102b70-cfa9-46b1-83f6-10f9126d22d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.373951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6102b70-cfa9-46b1-83f6-10f9126d22d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.374075 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaf82ea-4db5-4497-9b19-55bb9b1eb826 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.374277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6102b70-cfa9-46b1-83f6-10f9126d22d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.374974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6102b70-cfa9-46b1-83f6-10f9126d22d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.375924 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaf82ea-4db5-4497-9b19-55bb9b1eb826 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.377298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6102b70-cfa9-46b1-83f6-10f9126d22d4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.409586 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1346/2687] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:19 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:19.409829 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1342/2688] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:19 2022] POST /identity/v3/projects => generated 363 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:19.413456 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6102b70-cfa9-46b1-83f6-10f9126d22d4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.414093 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf82ea-4db5-4497-9b19-55bb9b1eb826 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.487550 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf82ea-4db5-4497-9b19-55bb9b1eb826 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.493690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea7116-e14b-4ba2-9bc1-2717ed4e4bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.494333 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea7116-e14b-4ba2-9bc1-2717ed4e4bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.494614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea7116-e14b-4ba2-9bc1-2717ed4e4bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.494886 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea7116-e14b-4ba2-9bc1-2717ed4e4bd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.495538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ea7116-e14b-4ba2-9bc1-2717ed4e4bd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.497633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ea7116-e14b-4ba2-9bc1-2717ed4e4bd7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.527446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1343/2689] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:19 2022] POST /identity/v3/users => generated 282 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:08:19.528502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea7116-e14b-4ba2-9bc1-2717ed4e4bd7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.540414 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea7116-e14b-4ba2-9bc1-2717ed4e4bd7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.545343 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b0e53-9485-4d01-8dea-40dd2bfc2a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.546008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b0e53-9485-4d01-8dea-40dd2bfc2a87 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.546291 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b0e53-9485-4d01-8dea-40dd2bfc2a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.546565 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b0e53-9485-4d01-8dea-40dd2bfc2a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.547231 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b0e53-9485-4d01-8dea-40dd2bfc2a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.549014 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b0e53-9485-4d01-8dea-40dd2bfc2a87 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.559925 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1344/2690] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:19.564159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b0e53-9485-4d01-8dea-40dd2bfc2a87 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.574308 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b0e53-9485-4d01-8dea-40dd2bfc2a87 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.580035 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5cc7a-2e11-410d-acc9-d2806b22095d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.580712 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5cc7a-2e11-410d-acc9-d2806b22095d admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.581003 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5cc7a-2e11-410d-acc9-d2806b22095d admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.581280 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5cc7a-2e11-410d-acc9-d2806b22095d admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.581932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5cc7a-2e11-410d-acc9-d2806b22095d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.583536 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5cc7a-2e11-410d-acc9-d2806b22095d admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.604961 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1345/2691] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:08:19.607175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5cc7a-2e11-410d-acc9-d2806b22095d admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.622139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5cc7a-2e11-410d-acc9-d2806b22095d admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.627776 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6102b70-cfa9-46b1-83f6-10f9126d22d4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.628894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6634e3-42f1-46e6-a90d-46ec7f9ef076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.629740 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6634e3-42f1-46e6-a90d-46ec7f9ef076 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.630094 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6634e3-42f1-46e6-a90d-46ec7f9ef076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.630304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6634e3-42f1-46e6-a90d-46ec7f9ef076 admin admin] PATH_INFO: `/v3/projects/06e7b212701d491592c1c49fb28faa81/users/f1765df8727c470abc3f55691b9c3c7d/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.633174 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5daaed7-da4b-44cf-ba95-45c44fa209fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.634067 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daaed7-da4b-44cf-ba95-45c44fa209fe admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.634480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daaed7-da4b-44cf-ba95-45c44fa209fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.634849 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daaed7-da4b-44cf-ba95-45c44fa209fe admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.635591 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5daaed7-da4b-44cf-ba95-45c44fa209fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.637752 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5daaed7-da4b-44cf-ba95-45c44fa209fe admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.647810 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6634e3-42f1-46e6-a90d-46ec7f9ef076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e7b212701d491592c1c49fb28faa81, user_id=f1765df8727c470abc3f55691b9c3c7d, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.650540 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6634e3-42f1-46e6-a90d-46ec7f9ef076 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.665573 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1346/2692] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:19 2022] PUT /identity/v3/projects/06e7b212701d491592c1c49fb28faa81/users/f1765df8727c470abc3f55691b9c3c7d/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:19.675589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1347/2693] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:19 2022] POST /identity/v3/users => generated 299 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:08:19.675589 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5daaed7-da4b-44cf-ba95-45c44fa209fe admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.677228 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad806c-4f69-44c1-9446-f269c79f82d1 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.677621 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad806c-4f69-44c1-9446-f269c79f82d1 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.677999 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad806c-4f69-44c1-9446-f269c79f82d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.681804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5daaed7-da4b-44cf-ba95-45c44fa209fe admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.713098 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-46ad806c-4f69-44c1-9446-f269c79f82d1 None None] MFA Rules not processed for user `f1765df8727c470abc3f55691b9c3c7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:19.714317 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-46ad806c-4f69-44c1-9446-f269c79f82d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:19.723498 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:19.752392 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a213044-0a89-4089-836e-0cb4fbcc9fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.752898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a213044-0a89-4089-836e-0cb4fbcc9fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.753829 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a213044-0a89-4089-836e-0cb4fbcc9fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.754133 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a213044-0a89-4089-836e-0cb4fbcc9fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.754848 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a213044-0a89-4089-836e-0cb4fbcc9fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.756411 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:19.756630 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a213044-0a89-4089-836e-0cb4fbcc9fb9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.771275 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1348/2694] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:19.772374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a213044-0a89-4089-836e-0cb4fbcc9fb9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.785912 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a213044-0a89-4089-836e-0cb4fbcc9fb9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.791622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cf4c4-d569-41fb-94c7-26c4bf700455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.794147 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cf4c4-d569-41fb-94c7-26c4bf700455 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.794506 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cf4c4-d569-41fb-94c7-26c4bf700455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.794779 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cf4c4-d569-41fb-94c7-26c4bf700455 admin admin] PATH_INFO: `/v3/projects/1be36f151720476b93775d03b89150d8/users/91ae235634e44c31bb728915d3d72d1f/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.805876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1347/2695] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:08:19.810344 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad806c-4f69-44c1-9446-f269c79f82d1 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.815470 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7cf4c4-d569-41fb-94c7-26c4bf700455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1be36f151720476b93775d03b89150d8, user_id=91ae235634e44c31bb728915d3d72d1f, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.818301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7cf4c4-d569-41fb-94c7-26c4bf700455 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.822257 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad806c-4f69-44c1-9446-f269c79f82d1 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.827306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1920fc-c3d1-4dc1-95dc-da9b9643a580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.827853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1920fc-c3d1-4dc1-95dc-da9b9643a580 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.828154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1920fc-c3d1-4dc1-95dc-da9b9643a580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.828731 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1920fc-c3d1-4dc1-95dc-da9b9643a580 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.829521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1920fc-c3d1-4dc1-95dc-da9b9643a580 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.830968 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1920fc-c3d1-4dc1-95dc-da9b9643a580 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.833590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1349/2696] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:19 2022] PUT /identity/v3/projects/1be36f151720476b93775d03b89150d8/users/91ae235634e44c31bb728915d3d72d1f/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:19.837194 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cf4c4-d569-41fb-94c7-26c4bf700455 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.870425 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1348/2697] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:19 2022] POST /identity/v3/projects => generated 363 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:08:19.877141 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1920fc-c3d1-4dc1-95dc-da9b9643a580 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.916273 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cf4c4-d569-41fb-94c7-26c4bf700455 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.944539 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1920fc-c3d1-4dc1-95dc-da9b9643a580 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.949848 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9468e2b-cd16-4d55-b059-dbc2d1c04082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.950461 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9468e2b-cd16-4d55-b059-dbc2d1c04082 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.950742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9468e2b-cd16-4d55-b059-dbc2d1c04082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.951018 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9468e2b-cd16-4d55-b059-dbc2d1c04082 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.951643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9468e2b-cd16-4d55-b059-dbc2d1c04082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.953585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9468e2b-cd16-4d55-b059-dbc2d1c04082 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.977696 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1349/2698] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:19 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:19.981809 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9468e2b-cd16-4d55-b059-dbc2d1c04082 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.986002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f6401-f54b-46ec-94b1-e867a60d0578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.986678 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f6401-f54b-46ec-94b1-e867a60d0578 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.986955 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f6401-f54b-46ec-94b1-e867a60d0578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.987226 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f6401-f54b-46ec-94b1-e867a60d0578 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.987995 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f6401-f54b-46ec-94b1-e867a60d0578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.989532 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f6401-f54b-46ec-94b1-e867a60d0578 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.994064 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9468e2b-cd16-4d55-b059-dbc2d1c04082 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.999867 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1350/2699] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:08:20.002278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d4e27-d5e6-43f8-bdcf-f398f428981a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.003123 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d4e27-d5e6-43f8-bdcf-f398f428981a admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.003536 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d4e27-d5e6-43f8-bdcf-f398f428981a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.003667 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d4e27-d5e6-43f8-bdcf-f398f428981a admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.004350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d4e27-d5e6-43f8-bdcf-f398f428981a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.005060 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f6401-f54b-46ec-94b1-e867a60d0578 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.006034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d4e27-d5e6-43f8-bdcf-f398f428981a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.018258 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f6401-f54b-46ec-94b1-e867a60d0578 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.019347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1350/2700] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:08:20.023116 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d4e27-d5e6-43f8-bdcf-f398f428981a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.026601 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95d514-0421-4e9c-aeea-8881ee550b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.027215 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95d514-0421-4e9c-aeea-8881ee550b64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.027493 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95d514-0421-4e9c-aeea-8881ee550b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.028691 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95d514-0421-4e9c-aeea-8881ee550b64 admin admin] PATH_INFO: `/v3/projects/1be36f151720476b93775d03b89150d8/users/91ae235634e44c31bb728915d3d72d1f/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.032179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95d514-0421-4e9c-aeea-8881ee550b64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1be36f151720476b93775d03b89150d8, user_id=91ae235634e44c31bb728915d3d72d1f, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.034964 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95d514-0421-4e9c-aeea-8881ee550b64 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.035472 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d4e27-d5e6-43f8-bdcf-f398f428981a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.040811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1275d-6188-4c79-a911-96013d481ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.041591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1275d-6188-4c79-a911-96013d481ce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.041925 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1275d-6188-4c79-a911-96013d481ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.042407 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1275d-6188-4c79-a911-96013d481ce9 admin admin] PATH_INFO: `/v3/projects/f9d1e00e6963452cbcfe81dc53c1c3a2/users/d0feff6c17064a0ca43f018572fb7e1a/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.052621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1351/2701] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:20 2022] PUT /identity/v3/projects/1be36f151720476b93775d03b89150d8/users/91ae235634e44c31bb728915d3d72d1f/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:20.058621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f65a0-b2b7-4e25-a658-e90b14bf8fcb None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.058902 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f65a0-b2b7-4e25-a658-e90b14bf8fcb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.059319 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f65a0-b2b7-4e25-a658-e90b14bf8fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.062895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1275d-6188-4c79-a911-96013d481ce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9d1e00e6963452cbcfe81dc53c1c3a2, user_id=d0feff6c17064a0ca43f018572fb7e1a, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.067260 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1275d-6188-4c79-a911-96013d481ce9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.082980 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1351/2702] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:20 2022] PUT /identity/v3/projects/f9d1e00e6963452cbcfe81dc53c1c3a2/users/d0feff6c17064a0ca43f018572fb7e1a/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:20.087039 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1275d-6188-4c79-a911-96013d481ce9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.091285 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5e3f65a0-b2b7-4e25-a658-e90b14bf8fcb None None] MFA Rules not processed for user `91ae235634e44c31bb728915d3d72d1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:20.092891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5e3f65a0-b2b7-4e25-a658-e90b14bf8fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:20.127441 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:20.176570 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1275d-6188-4c79-a911-96013d481ce9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.182211 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036f2bd-7557-4ce7-b377-f5c304374da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.182806 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036f2bd-7557-4ce7-b377-f5c304374da7 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.183084 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036f2bd-7557-4ce7-b377-f5c304374da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.183447 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036f2bd-7557-4ce7-b377-f5c304374da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.184151 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036f2bd-7557-4ce7-b377-f5c304374da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.187350 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036f2bd-7557-4ce7-b377-f5c304374da7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.189620 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:20.208666 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1352/2703] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:20 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:08:20.209712 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1352/2704] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:20.209847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f65a0-b2b7-4e25-a658-e90b14bf8fcb None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.224373 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f65a0-b2b7-4e25-a658-e90b14bf8fcb None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.225873 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036f2bd-7557-4ce7-b377-f5c304374da7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.228508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e036f2bd-7557-4ce7-b377-f5c304374da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:20.236727 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23be37-e6db-47e0-b112-e5341e7c7a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.237880 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23be37-e6db-47e0-b112-e5341e7c7a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.241039 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23be37-e6db-47e0-b112-e5341e7c7a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.241625 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23be37-e6db-47e0-b112-e5341e7c7a6b admin admin] PATH_INFO: `/v3/projects/f9d1e00e6963452cbcfe81dc53c1c3a2/users/d0feff6c17064a0ca43f018572fb7e1a/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.245365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23be37-e6db-47e0-b112-e5341e7c7a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9d1e00e6963452cbcfe81dc53c1c3a2, user_id=d0feff6c17064a0ca43f018572fb7e1a, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.248082 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23be37-e6db-47e0-b112-e5341e7c7a6b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.268142 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1353/2705] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:20 2022] PUT /identity/v3/projects/f9d1e00e6963452cbcfe81dc53c1c3a2/users/d0feff6c17064a0ca43f018572fb7e1a/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:20.268142 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23be37-e6db-47e0-b112-e5341e7c7a6b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.273191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-eb23be37-e6db-47e0-b112-e5341e7c7a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:20.292195 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036f2bd-7557-4ce7-b377-f5c304374da7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.339621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23be37-e6db-47e0-b112-e5341e7c7a6b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.341585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788aad8-c3fc-42e4-a81f-b2085bd4239f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.342276 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788aad8-c3fc-42e4-a81f-b2085bd4239f service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.342557 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788aad8-c3fc-42e4-a81f-b2085bd4239f service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.342831 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788aad8-c3fc-42e4-a81f-b2085bd4239f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.345215 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3fe92-0aa6-4760-aa94-9724d7946404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.345847 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3fe92-0aa6-4760-aa94-9724d7946404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.346120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3fe92-0aa6-4760-aa94-9724d7946404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.346393 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3fe92-0aa6-4760-aa94-9724d7946404 admin admin] PATH_INFO: `/v3/users/a72949c143d74b7c986e53ffb57945ab` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.347528 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3fe92-0aa6-4760-aa94-9724d7946404 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a72949c143d74b7c986e53ffb57945ab)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.349464 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3fe92-0aa6-4760-aa94-9724d7946404 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.395656 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-daa3fe92-0aa6-4760-aa94-9724d7946404 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a72949c143d74b7c986e53ffb57945ab'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:20.396336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-daa3fe92-0aa6-4760-aa94-9724d7946404 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a72949c143d74b7c986e53ffb57945ab'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:20.403585 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-daa3fe92-0aa6-4760-aa94-9724d7946404 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a72949c143d74b7c986e53ffb57945ab'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:20.407992 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788aad8-c3fc-42e4-a81f-b2085bd4239f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.409575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788aad8-c3fc-42e4-a81f-b2085bd4239f service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.411152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-daa3fe92-0aa6-4760-aa94-9724d7946404 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a72949c143d74b7c986e53ffb57945ab'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:20.419159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1353/2706] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:20.420396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda016d-89c0-4b1c-9e51-b35549b344d5 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.420682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda016d-89c0-4b1c-9e51-b35549b344d5 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.420941 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda016d-89c0-4b1c-9e51-b35549b344d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.424664 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-daa3fe92-0aa6-4760-aa94-9724d7946404 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a72949c143d74b7c986e53ffb57945ab'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:20.431335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1354/2707] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:20 2022] DELETE /identity/v3/users/a72949c143d74b7c986e53ffb57945ab => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:20.453273 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3fe92-0aa6-4760-aa94-9724d7946404 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.455451 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-daa3fe92-0aa6-4760-aa94-9724d7946404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:20.455997 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-2eda016d-89c0-4b1c-9e51-b35549b344d5 None None] MFA Rules not processed for user `d0feff6c17064a0ca43f018572fb7e1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:20.457215 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2eda016d-89c0-4b1c-9e51-b35549b344d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:20.496401 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:20.515404 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:20.541008 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1354/2708] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:20.541243 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3fe92-0aa6-4760-aa94-9724d7946404 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.548341 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a999dd8-4411-42ce-b31d-d5a6fa85e789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.548993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a999dd8-4411-42ce-b31d-d5a6fa85e789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.549267 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a999dd8-4411-42ce-b31d-d5a6fa85e789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.549535 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a999dd8-4411-42ce-b31d-d5a6fa85e789 admin admin] PATH_INFO: `/v3/users/6ae9e8fadd7e4d57a0bc9b10e3cb0b54` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.552190 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a999dd8-4411-42ce-b31d-d5a6fa85e789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ae9e8fadd7e4d57a0bc9b10e3cb0b54)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.553430 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda016d-89c0-4b1c-9e51-b35549b344d5 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.555135 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a999dd8-4411-42ce-b31d-d5a6fa85e789 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.555241 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2eda016d-89c0-4b1c-9e51-b35549b344d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:20.612354 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9a999dd8-4411-42ce-b31d-d5a6fa85e789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ae9e8fadd7e4d57a0bc9b10e3cb0b54'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:20.613095 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9a999dd8-4411-42ce-b31d-d5a6fa85e789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ae9e8fadd7e4d57a0bc9b10e3cb0b54'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:20.620777 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9a999dd8-4411-42ce-b31d-d5a6fa85e789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ae9e8fadd7e4d57a0bc9b10e3cb0b54'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:20.629304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda016d-89c0-4b1c-9e51-b35549b344d5 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.631665 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9a999dd8-4411-42ce-b31d-d5a6fa85e789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ae9e8fadd7e4d57a0bc9b10e3cb0b54'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:20.633375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd88d86f-471f-48ee-ab8f-0320d355b01b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.634016 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88d86f-471f-48ee-ab8f-0320d355b01b service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.634275 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88d86f-471f-48ee-ab8f-0320d355b01b service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.634546 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88d86f-471f-48ee-ab8f-0320d355b01b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.636445 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-bd88d86f-471f-48ee-ab8f-0320d355b01b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:20.650099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd88d86f-471f-48ee-ab8f-0320d355b01b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.651577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd88d86f-471f-48ee-ab8f-0320d355b01b service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.652050 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9a999dd8-4411-42ce-b31d-d5a6fa85e789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ae9e8fadd7e4d57a0bc9b10e3cb0b54'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:20.659265 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1355/2709] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:20 2022] DELETE /identity/v3/users/6ae9e8fadd7e4d57a0bc9b10e3cb0b54 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:20.660188 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a999dd8-4411-42ce-b31d-d5a6fa85e789 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.662079 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9a999dd8-4411-42ce-b31d-d5a6fa85e789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:20.667715 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1355/2710] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:20 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:20.669042 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09209ee6-d433-4038-87e0-99bec552f401 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.669835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09209ee6-d433-4038-87e0-99bec552f401 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.670173 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09209ee6-d433-4038-87e0-99bec552f401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.693012 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-09209ee6-d433-4038-87e0-99bec552f401 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:20.694361 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-09209ee6-d433-4038-87e0-99bec552f401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:20.739880 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a999dd8-4411-42ce-b31d-d5a6fa85e789 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.746655 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311011b8-0437-4c86-a1ee-95e9e78e5302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.747358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311011b8-0437-4c86-a1ee-95e9e78e5302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.747639 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311011b8-0437-4c86-a1ee-95e9e78e5302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.747908 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311011b8-0437-4c86-a1ee-95e9e78e5302 admin admin] PATH_INFO: `/v3/projects/6ae11287e9794e0f967b07b9b7bd6ffb` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.749091 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311011b8-0437-4c86-a1ee-95e9e78e5302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ae11287e9794e0f967b07b9b7bd6ffb)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.751583 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311011b8-0437-4c86-a1ee-95e9e78e5302 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.780944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1356/2711] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:20.791298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09209ee6-d433-4038-87e0-99bec552f401 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.792796 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-09209ee6-d433-4038-87e0-99bec552f401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:20.815535 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-311011b8-0437-4c86-a1ee-95e9e78e5302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ae11287e9794e0f967b07b9b7bd6ffb', 'request_id': 'req-311011b8-0437-4c86-a1ee-95e9e78e5302'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:20.816912 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:20.822743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-311011b8-0437-4c86-a1ee-95e9e78e5302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ae11287e9794e0f967b07b9b7bd6ffb', 'request_id': 'req-311011b8-0437-4c86-a1ee-95e9e78e5302'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:20.832167 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-311011b8-0437-4c86-a1ee-95e9e78e5302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ae11287e9794e0f967b07b9b7bd6ffb', 'request_id': 'req-311011b8-0437-4c86-a1ee-95e9e78e5302'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:20.848697 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-311011b8-0437-4c86-a1ee-95e9e78e5302 admin admin] The token cache is being invalidate because project 6ae11287e9794e0f967b07b9b7bd6ffb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:20.849683 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-311011b8-0437-4c86-a1ee-95e9e78e5302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:20.850522 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-311011b8-0437-4c86-a1ee-95e9e78e5302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:20.856193 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1356/2712] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:20 2022] DELETE /identity/v3/projects/6ae11287e9794e0f967b07b9b7bd6ffb => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:20.857669 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311011b8-0437-4c86-a1ee-95e9e78e5302 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.859129 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-311011b8-0437-4c86-a1ee-95e9e78e5302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:20.869940 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09209ee6-d433-4038-87e0-99bec552f401 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.921565 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:20.931488 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75450e3d-b40e-4d7f-b9bb-5cb25068d0b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.932146 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75450e3d-b40e-4d7f-b9bb-5cb25068d0b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.932416 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75450e3d-b40e-4d7f-b9bb-5cb25068d0b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.932695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75450e3d-b40e-4d7f-b9bb-5cb25068d0b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.934535 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-75450e3d-b40e-4d7f-b9bb-5cb25068d0b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:20.943893 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311011b8-0437-4c86-a1ee-95e9e78e5302 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.949162 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75450e3d-b40e-4d7f-b9bb-5cb25068d0b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.950604 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75450e3d-b40e-4d7f-b9bb-5cb25068d0b5 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.954292 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e19d7-ed2c-4d01-b1ad-72f2988e2852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.955543 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e19d7-ed2c-4d01-b1ad-72f2988e2852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.955908 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e19d7-ed2c-4d01-b1ad-72f2988e2852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.956245 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e19d7-ed2c-4d01-b1ad-72f2988e2852 admin admin] PATH_INFO: `/v3/users/a48ce95e8c754853865fa982a3b2b97d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.957472 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e19d7-ed2c-4d01-b1ad-72f2988e2852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a48ce95e8c754853865fa982a3b2b97d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.959780 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e19d7-ed2c-4d01-b1ad-72f2988e2852 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.963604 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1357/2713] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:08:21.021615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-326e19d7-ed2c-4d01-b1ad-72f2988e2852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a48ce95e8c754853865fa982a3b2b97d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.022805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-326e19d7-ed2c-4d01-b1ad-72f2988e2852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a48ce95e8c754853865fa982a3b2b97d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.034016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-326e19d7-ed2c-4d01-b1ad-72f2988e2852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a48ce95e8c754853865fa982a3b2b97d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.048256 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-326e19d7-ed2c-4d01-b1ad-72f2988e2852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a48ce95e8c754853865fa982a3b2b97d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.057622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-326e19d7-ed2c-4d01-b1ad-72f2988e2852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a48ce95e8c754853865fa982a3b2b97d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.067413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1357/2714] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:20 2022] DELETE /identity/v3/users/a48ce95e8c754853865fa982a3b2b97d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:21.154144 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75450e3d-b40e-4d7f-b9bb-5cb25068d0b5 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.156367 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-75450e3d-b40e-4d7f-b9bb-5cb25068d0b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:21.169980 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75450e3d-b40e-4d7f-b9bb-5cb25068d0b5 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.236975 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f74fa-023a-4bae-87d0-083ed4d764b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.237756 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f74fa-023a-4bae-87d0-083ed4d764b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.238119 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f74fa-023a-4bae-87d0-083ed4d764b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.238463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f74fa-023a-4bae-87d0-083ed4d764b1 admin admin] PATH_INFO: `/v3/users/f1765df8727c470abc3f55691b9c3c7d` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.239776 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f74fa-023a-4bae-87d0-083ed4d764b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1765df8727c470abc3f55691b9c3c7d)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.241936 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f74fa-023a-4bae-87d0-083ed4d764b1 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.262325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e19d7-ed2c-4d01-b1ad-72f2988e2852 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.264450 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-326e19d7-ed2c-4d01-b1ad-72f2988e2852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:21.281350 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e19d7-ed2c-4d01-b1ad-72f2988e2852 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.292050 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39a52e-7c19-4b8d-a24d-206cd7419f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.292636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39a52e-7c19-4b8d-a24d-206cd7419f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.293206 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39a52e-7c19-4b8d-a24d-206cd7419f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.293633 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39a52e-7c19-4b8d-a24d-206cd7419f37 admin admin] PATH_INFO: `/v3/projects/7f951df5d72a44c19ba472eaf6278665` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.294961 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39a52e-7c19-4b8d-a24d-206cd7419f37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f951df5d72a44c19ba472eaf6278665)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.297203 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39a52e-7c19-4b8d-a24d-206cd7419f37 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.332380 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-696f74fa-023a-4bae-87d0-083ed4d764b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1765df8727c470abc3f55691b9c3c7d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.342794 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-696f74fa-023a-4bae-87d0-083ed4d764b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1765df8727c470abc3f55691b9c3c7d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.352330 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fb39a52e-7c19-4b8d-a24d-206cd7419f37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f951df5d72a44c19ba472eaf6278665', 'request_id': 'req-fb39a52e-7c19-4b8d-a24d-206cd7419f37'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.354380 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:21.356273 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-696f74fa-023a-4bae-87d0-083ed4d764b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1765df8727c470abc3f55691b9c3c7d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.359992 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fb39a52e-7c19-4b8d-a24d-206cd7419f37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f951df5d72a44c19ba472eaf6278665', 'request_id': 'req-fb39a52e-7c19-4b8d-a24d-206cd7419f37'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.363707 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-696f74fa-023a-4bae-87d0-083ed4d764b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1765df8727c470abc3f55691b9c3c7d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.364421 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-696f74fa-023a-4bae-87d0-083ed4d764b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1765df8727c470abc3f55691b9c3c7d'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.369348 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fb39a52e-7c19-4b8d-a24d-206cd7419f37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f951df5d72a44c19ba472eaf6278665', 'request_id': 'req-fb39a52e-7c19-4b8d-a24d-206cd7419f37'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.370727 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1358/2715] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:21 2022] DELETE /identity/v3/users/f1765df8727c470abc3f55691b9c3c7d => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:21.371502 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f74fa-023a-4bae-87d0-083ed4d764b1 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.372828 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-696f74fa-023a-4bae-87d0-083ed4d764b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:21.384454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fb39a52e-7c19-4b8d-a24d-206cd7419f37 admin admin] The token cache is being invalidate because project 7f951df5d72a44c19ba472eaf6278665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:21.384741 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fb39a52e-7c19-4b8d-a24d-206cd7419f37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.385380 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fb39a52e-7c19-4b8d-a24d-206cd7419f37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.386882 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1358/2716] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:21 2022] DELETE /identity/v3/projects/7f951df5d72a44c19ba472eaf6278665 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:21.388036 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45876103-8e6b-4bd8-872b-5d6b8ca3656b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.388342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45876103-8e6b-4bd8-872b-5d6b8ca3656b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.388613 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45876103-8e6b-4bd8-872b-5d6b8ca3656b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.407517 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-45876103-8e6b-4bd8-872b-5d6b8ca3656b None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:21.408670 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-45876103-8e6b-4bd8-872b-5d6b8ca3656b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:21.443341 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f74fa-023a-4bae-87d0-083ed4d764b1 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.448249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615ce58-9608-4cb9-b3d8-4564c04293ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.448890 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615ce58-9608-4cb9-b3d8-4564c04293ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.449173 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615ce58-9608-4cb9-b3d8-4564c04293ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.449442 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615ce58-9608-4cb9-b3d8-4564c04293ac admin admin] PATH_INFO: `/v3/projects/ccd191c8d6504611a7985af2ecac5dbb` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.450571 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615ce58-9608-4cb9-b3d8-4564c04293ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccd191c8d6504611a7985af2ecac5dbb)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.452509 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615ce58-9608-4cb9-b3d8-4564c04293ac admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.464054 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:21.467161 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1359/2717] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:21.468455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2434825-a8ee-494d-9e18-bd02ad69207c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.468733 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2434825-a8ee-494d-9e18-bd02ad69207c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.469661 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2434825-a8ee-494d-9e18-bd02ad69207c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.490090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b2434825-a8ee-494d-9e18-bd02ad69207c None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:21.491356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b2434825-a8ee-494d-9e18-bd02ad69207c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:21.506895 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4615ce58-9608-4cb9-b3d8-4564c04293ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccd191c8d6504611a7985af2ecac5dbb', 'request_id': 'req-4615ce58-9608-4cb9-b3d8-4564c04293ac'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.514630 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4615ce58-9608-4cb9-b3d8-4564c04293ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccd191c8d6504611a7985af2ecac5dbb', 'request_id': 'req-4615ce58-9608-4cb9-b3d8-4564c04293ac'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.520399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4615ce58-9608-4cb9-b3d8-4564c04293ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccd191c8d6504611a7985af2ecac5dbb', 'request_id': 'req-4615ce58-9608-4cb9-b3d8-4564c04293ac'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.532069 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4615ce58-9608-4cb9-b3d8-4564c04293ac admin admin] The token cache is being invalidate because project ccd191c8d6504611a7985af2ecac5dbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:21.532382 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4615ce58-9608-4cb9-b3d8-4564c04293ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.533118 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-4615ce58-9608-4cb9-b3d8-4564c04293ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.534665 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1359/2718] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:21 2022] DELETE /identity/v3/projects/ccd191c8d6504611a7985af2ecac5dbb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:21.535479 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615ce58-9608-4cb9-b3d8-4564c04293ac admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.536844 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4615ce58-9608-4cb9-b3d8-4564c04293ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:21.552484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1360/2719] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:21.553321 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2434825-a8ee-494d-9e18-bd02ad69207c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.554676 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b2434825-a8ee-494d-9e18-bd02ad69207c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:21.566329 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2434825-a8ee-494d-9e18-bd02ad69207c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.571596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210ab8b-5091-40ff-88ad-b49bf33baa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.572260 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210ab8b-5091-40ff-88ad-b49bf33baa0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.572537 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210ab8b-5091-40ff-88ad-b49bf33baa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.572827 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210ab8b-5091-40ff-88ad-b49bf33baa0e admin admin] PATH_INFO: `/v3/users/91ae235634e44c31bb728915d3d72d1f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.573949 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210ab8b-5091-40ff-88ad-b49bf33baa0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=91ae235634e44c31bb728915d3d72d1f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.577249 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210ab8b-5091-40ff-88ad-b49bf33baa0e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.608433 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615ce58-9608-4cb9-b3d8-4564c04293ac admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.612823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c04a5-d11e-4896-84a3-ea13697d3eba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.613439 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c04a5-d11e-4896-84a3-ea13697d3eba service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.613720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c04a5-d11e-4896-84a3-ea13697d3eba service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.613988 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c04a5-d11e-4896-84a3-ea13697d3eba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.615727 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-595c04a5-d11e-4896-84a3-ea13697d3eba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:21.626632 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f210ab8b-5091-40ff-88ad-b49bf33baa0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91ae235634e44c31bb728915d3d72d1f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.627335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f210ab8b-5091-40ff-88ad-b49bf33baa0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91ae235634e44c31bb728915d3d72d1f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.635314 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c04a5-d11e-4896-84a3-ea13697d3eba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.636830 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c04a5-d11e-4896-84a3-ea13697d3eba service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.640534 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f210ab8b-5091-40ff-88ad-b49bf33baa0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91ae235634e44c31bb728915d3d72d1f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.647842 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1360/2720] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:21.648269 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f210ab8b-5091-40ff-88ad-b49bf33baa0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91ae235634e44c31bb728915d3d72d1f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.649319 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d24ccd-4829-4355-8aa2-a4c3203e9a7c None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.649611 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d24ccd-4829-4355-8aa2-a4c3203e9a7c None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.649904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d24ccd-4829-4355-8aa2-a4c3203e9a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.661394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:21.664129 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f210ab8b-5091-40ff-88ad-b49bf33baa0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91ae235634e44c31bb728915d3d72d1f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:21.671176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-63d24ccd-4829-4355-8aa2-a4c3203e9a7c None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:21.673130 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1361/2721] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:21 2022] DELETE /identity/v3/users/91ae235634e44c31bb728915d3d72d1f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:21.675997 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-63d24ccd-4829-4355-8aa2-a4c3203e9a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:21.767901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1361/2722] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:21.769046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210ab8b-5091-40ff-88ad-b49bf33baa0e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.780357 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210ab8b-5091-40ff-88ad-b49bf33baa0e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.787207 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241000f-8ffd-404d-8d60-cc27cfdea109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.787913 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241000f-8ffd-404d-8d60-cc27cfdea109 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.788296 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241000f-8ffd-404d-8d60-cc27cfdea109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.788635 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241000f-8ffd-404d-8d60-cc27cfdea109 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.789358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d241000f-8ffd-404d-8d60-cc27cfdea109 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:21.789756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d241000f-8ffd-404d-8d60-cc27cfdea109 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.791407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d241000f-8ffd-404d-8d60-cc27cfdea109 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.806241 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1362/2723] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:21.810805 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d24ccd-4829-4355-8aa2-a4c3203e9a7c None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.820652 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d24ccd-4829-4355-8aa2-a4c3203e9a7c None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.829161 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd7b04-098e-49cc-a69b-d4c15a634990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.829801 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd7b04-098e-49cc-a69b-d4c15a634990 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.830079 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd7b04-098e-49cc-a69b-d4c15a634990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.830538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd7b04-098e-49cc-a69b-d4c15a634990 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.831050 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd7b04-098e-49cc-a69b-d4c15a634990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.833204 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd7b04-098e-49cc-a69b-d4c15a634990 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.876043 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1362/2724] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:21 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:08:21.877511 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241000f-8ffd-404d-8d60-cc27cfdea109 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.887836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241000f-8ffd-404d-8d60-cc27cfdea109 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.965444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52e814-987d-4468-833a-bfae36652f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.966315 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52e814-987d-4468-833a-bfae36652f0b admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.966590 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52e814-987d-4468-833a-bfae36652f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.967301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52e814-987d-4468-833a-bfae36652f0b admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.968121 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf52e814-987d-4468-833a-bfae36652f0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.971285 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf52e814-987d-4468-833a-bfae36652f0b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.987570 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd7b04-098e-49cc-a69b-d4c15a634990 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.989686 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2efd7b04-098e-49cc-a69b-d4c15a634990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:22.013713 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1363/2725] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:21 2022] POST /identity/v3/users => generated 286 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:22.014781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52e814-987d-4468-833a-bfae36652f0b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:22.017043 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cf52e814-987d-4468-833a-bfae36652f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:22.017648 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd7b04-098e-49cc-a69b-d4c15a634990 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:22.022864 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c688f6-005f-4442-9e7b-66a64cba3055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:22.023579 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c688f6-005f-4442-9e7b-66a64cba3055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.023931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c688f6-005f-4442-9e7b-66a64cba3055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.024298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c688f6-005f-4442-9e7b-66a64cba3055 admin admin] PATH_INFO: `/v3/projects/06e7b212701d491592c1c49fb28faa81` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.025507 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c688f6-005f-4442-9e7b-66a64cba3055 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06e7b212701d491592c1c49fb28faa81)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:22.027524 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c688f6-005f-4442-9e7b-66a64cba3055 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:22.033686 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52e814-987d-4468-833a-bfae36652f0b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:22.040050 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ba808-7383-4181-b721-51fc97fcadc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:22.040745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ba808-7383-4181-b721-51fc97fcadc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.041127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ba808-7383-4181-b721-51fc97fcadc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.041604 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ba808-7383-4181-b721-51fc97fcadc0 admin admin] PATH_INFO: `/v3/projects/1be36f151720476b93775d03b89150d8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.042898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ba808-7383-4181-b721-51fc97fcadc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1be36f151720476b93775d03b89150d8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:22.045219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ba808-7383-4181-b721-51fc97fcadc0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:22.048933 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:22.086109 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f6c688f6-005f-4442-9e7b-66a64cba3055 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06e7b212701d491592c1c49fb28faa81', 'request_id': 'req-f6c688f6-005f-4442-9e7b-66a64cba3055'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.094538 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f6c688f6-005f-4442-9e7b-66a64cba3055 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06e7b212701d491592c1c49fb28faa81', 'request_id': 'req-f6c688f6-005f-4442-9e7b-66a64cba3055'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.098361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8e1ba808-7383-4181-b721-51fc97fcadc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1be36f151720476b93775d03b89150d8', 'request_id': 'req-8e1ba808-7383-4181-b721-51fc97fcadc0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.100962 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f6c688f6-005f-4442-9e7b-66a64cba3055 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06e7b212701d491592c1c49fb28faa81', 'request_id': 'req-f6c688f6-005f-4442-9e7b-66a64cba3055'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.104065 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8e1ba808-7383-4181-b721-51fc97fcadc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1be36f151720476b93775d03b89150d8', 'request_id': 'req-8e1ba808-7383-4181-b721-51fc97fcadc0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.112929 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8e1ba808-7383-4181-b721-51fc97fcadc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1be36f151720476b93775d03b89150d8', 'request_id': 'req-8e1ba808-7383-4181-b721-51fc97fcadc0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.113311 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f6c688f6-005f-4442-9e7b-66a64cba3055 admin admin] The token cache is being invalidate because project 06e7b212701d491592c1c49fb28faa81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:22.113601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f6c688f6-005f-4442-9e7b-66a64cba3055 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.114283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-f6c688f6-005f-4442-9e7b-66a64cba3055 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.115810 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1363/2726] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:21 2022] DELETE /identity/v3/projects/06e7b212701d491592c1c49fb28faa81 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:22.119426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c688f6-005f-4442-9e7b-66a64cba3055 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:22.120761 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f6c688f6-005f-4442-9e7b-66a64cba3055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:22.124834 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8e1ba808-7383-4181-b721-51fc97fcadc0 admin admin] The token cache is being invalidate because project 1be36f151720476b93775d03b89150d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:22.125802 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8e1ba808-7383-4181-b721-51fc97fcadc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.126601 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8e1ba808-7383-4181-b721-51fc97fcadc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.128109 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1364/2727] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:22 2022] DELETE /identity/v3/projects/1be36f151720476b93775d03b89150d8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:22.128878 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ba808-7383-4181-b721-51fc97fcadc0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:22.130132 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8e1ba808-7383-4181-b721-51fc97fcadc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:22.182549 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:22.186653 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c688f6-005f-4442-9e7b-66a64cba3055 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:22.192493 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d56004-fb34-4336-af1d-cb4f9fc45faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:22.193330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d56004-fb34-4336-af1d-cb4f9fc45faa admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.193612 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d56004-fb34-4336-af1d-cb4f9fc45faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.193916 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d56004-fb34-4336-af1d-cb4f9fc45faa admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.194762 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d56004-fb34-4336-af1d-cb4f9fc45faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:22.196045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ba808-7383-4181-b721-51fc97fcadc0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:22.196395 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d56004-fb34-4336-af1d-cb4f9fc45faa admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:22.203906 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb20cd-c749-4c04-b58d-662e1c32b521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:22.204891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb20cd-c749-4c04-b58d-662e1c32b521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.205250 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb20cd-c749-4c04-b58d-662e1c32b521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.205529 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb20cd-c749-4c04-b58d-662e1c32b521 admin admin] PATH_INFO: `/v3/users/d0feff6c17064a0ca43f018572fb7e1a` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.206756 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb20cd-c749-4c04-b58d-662e1c32b521 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0feff6c17064a0ca43f018572fb7e1a)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:22.207906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1364/2728] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:22.208738 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb20cd-c749-4c04-b58d-662e1c32b521 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:22.209172 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb4e86-71c7-460e-94ed-62bb21673fd2 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.209458 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb4e86-71c7-460e-94ed-62bb21673fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.209734 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb4e86-71c7-460e-94ed-62bb21673fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.230267 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-07eb4e86-71c7-460e-94ed-62bb21673fd2 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:22.231549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-07eb4e86-71c7-460e-94ed-62bb21673fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:22.241033 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1365/2729] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:22.241931 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb4e86-71c7-460e-94ed-62bb21673fd2 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:22.243365 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-07eb4e86-71c7-460e-94ed-62bb21673fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:22.256929 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb4e86-71c7-460e-94ed-62bb21673fd2 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:22.259355 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6acb20cd-c749-4c04-b58d-662e1c32b521 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0feff6c17064a0ca43f018572fb7e1a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.260044 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6acb20cd-c749-4c04-b58d-662e1c32b521 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0feff6c17064a0ca43f018572fb7e1a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.262913 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dcbbb-751c-432d-94b0-5686798c95fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:22.263516 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dcbbb-751c-432d-94b0-5686798c95fd admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.263793 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dcbbb-751c-432d-94b0-5686798c95fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.264203 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dcbbb-751c-432d-94b0-5686798c95fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.264882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482dcbbb-751c-432d-94b0-5686798c95fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:22.267612 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482dcbbb-751c-432d-94b0-5686798c95fd admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:22.269021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6acb20cd-c749-4c04-b58d-662e1c32b521 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0feff6c17064a0ca43f018572fb7e1a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.276419 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6acb20cd-c749-4c04-b58d-662e1c32b521 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0feff6c17064a0ca43f018572fb7e1a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.281908 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1366/2730] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:22.281908 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15431ae-6646-4dc2-b1d3-a034e50a9cf0 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.281908 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15431ae-6646-4dc2-b1d3-a034e50a9cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.281908 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15431ae-6646-4dc2-b1d3-a034e50a9cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.289265 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6acb20cd-c749-4c04-b58d-662e1c32b521 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0feff6c17064a0ca43f018572fb7e1a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.295307 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1365/2731] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:22 2022] DELETE /identity/v3/users/d0feff6c17064a0ca43f018572fb7e1a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:22.295998 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb20cd-c749-4c04-b58d-662e1c32b521 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:22.298802 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6acb20cd-c749-4c04-b58d-662e1c32b521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:22.303307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-e15431ae-6646-4dc2-b1d3-a034e50a9cf0 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:22.304730 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e15431ae-6646-4dc2-b1d3-a034e50a9cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:22.376046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb20cd-c749-4c04-b58d-662e1c32b521 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:22.382562 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf649e-48b4-442f-920a-55cf9b2f3e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:22.383238 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf649e-48b4-442f-920a-55cf9b2f3e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.383518 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf649e-48b4-442f-920a-55cf9b2f3e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.383830 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf649e-48b4-442f-920a-55cf9b2f3e0e admin admin] PATH_INFO: `/v3/projects/c4c91799ea4b4de1b8e0ba9229d8cc18/users/bf3d32b7b49945cea550f13849bf9995/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.389063 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1367/2732] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:22.389805 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15431ae-6646-4dc2-b1d3-a034e50a9cf0 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:22.401118 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15431ae-6646-4dc2-b1d3-a034e50a9cf0 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:22.401499 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf649e-48b4-442f-920a-55cf9b2f3e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4c91799ea4b4de1b8e0ba9229d8cc18, user_id=bf3d32b7b49945cea550f13849bf9995, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:22.404406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf649e-48b4-442f-920a-55cf9b2f3e0e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:22.407927 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263f7cc-6ded-48d6-9403-b0d298fa5f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:22.408161 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263f7cc-6ded-48d6-9403-b0d298fa5f4a admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.408522 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263f7cc-6ded-48d6-9403-b0d298fa5f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.408875 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263f7cc-6ded-48d6-9403-b0d298fa5f4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.409584 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263f7cc-6ded-48d6-9403-b0d298fa5f4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:22.409929 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263f7cc-6ded-48d6-9403-b0d298fa5f4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:22.411621 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263f7cc-6ded-48d6-9403-b0d298fa5f4a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:22.420285 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1366/2733] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:22 2022] PUT /identity/v3/projects/c4c91799ea4b4de1b8e0ba9229d8cc18/users/bf3d32b7b49945cea550f13849bf9995/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:22.425943 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1368/2734] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:22.430099 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263f7cc-6ded-48d6-9403-b0d298fa5f4a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:22.430436 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f8dc1-63c9-447a-bf59-fb6e6182ff17 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.430751 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f8dc1-63c9-447a-bf59-fb6e6182ff17 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.431028 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f8dc1-63c9-447a-bf59-fb6e6182ff17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.444045 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263f7cc-6ded-48d6-9403-b0d298fa5f4a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:22.450215 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:22.462149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-201f8dc1-63c9-447a-bf59-fb6e6182ff17 None None] MFA Rules not processed for user `bf3d32b7b49945cea550f13849bf9995`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:22.463297 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-201f8dc1-63c9-447a-bf59-fb6e6182ff17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:22.516597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:22.517146 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545bab4-07a8-4114-9b21-c2386be56ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:22.518029 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545bab4-07a8-4114-9b21-c2386be56ed9 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.518545 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545bab4-07a8-4114-9b21-c2386be56ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.519043 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545bab4-07a8-4114-9b21-c2386be56ed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.519675 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545bab4-07a8-4114-9b21-c2386be56ed9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:22.521852 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545bab4-07a8-4114-9b21-c2386be56ed9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:22.543443 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1367/2735] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:08:22.556226 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f8dc1-63c9-447a-bf59-fb6e6182ff17 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:22.556731 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1369/2736] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:22 2022] POST /identity/v3/projects => generated 393 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:22.559609 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-201f8dc1-63c9-447a-bf59-fb6e6182ff17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:22.561980 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545bab4-07a8-4114-9b21-c2386be56ed9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:22.639052 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f8dc1-63c9-447a-bf59-fb6e6182ff17 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:22.639232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545bab4-07a8-4114-9b21-c2386be56ed9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:22.645097 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b8e49-be0f-49e0-9cda-df75e5b21fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:22.645549 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b8e49-be0f-49e0-9cda-df75e5b21fa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.645826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b8e49-be0f-49e0-9cda-df75e5b21fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.646097 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b8e49-be0f-49e0-9cda-df75e5b21fa7 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.646728 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b8e49-be0f-49e0-9cda-df75e5b21fa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:22.648723 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b8e49-be0f-49e0-9cda-df75e5b21fa7 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:22.679072 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1370/2737] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:22 2022] POST /identity/v3/users => generated 297 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:22.680032 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b8e49-be0f-49e0-9cda-df75e5b21fa7 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:22.681648 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-812b8e49-be0f-49e0-9cda-df75e5b21fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:22.693996 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b8e49-be0f-49e0-9cda-df75e5b21fa7 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:22.698642 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5e632-8b41-4460-b649-808fc8c9fded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:22.699345 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5e632-8b41-4460-b649-808fc8c9fded admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.699695 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5e632-8b41-4460-b649-808fc8c9fded admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.700029 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5e632-8b41-4460-b649-808fc8c9fded admin admin] PATH_INFO: `/v3/projects/f9d1e00e6963452cbcfe81dc53c1c3a2` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.701249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5e632-8b41-4460-b649-808fc8c9fded admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9d1e00e6963452cbcfe81dc53c1c3a2)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:22.702728 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a417533-adfb-4a29-92bf-658a459edef9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:22.703698 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5e632-8b41-4460-b649-808fc8c9fded admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:22.704189 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a417533-adfb-4a29-92bf-658a459edef9 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.704474 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a417533-adfb-4a29-92bf-658a459edef9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.704748 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a417533-adfb-4a29-92bf-658a459edef9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.754469 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-68e5e632-8b41-4460-b649-808fc8c9fded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9d1e00e6963452cbcfe81dc53c1c3a2', 'request_id': 'req-68e5e632-8b41-4460-b649-808fc8c9fded'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.764069 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-68e5e632-8b41-4460-b649-808fc8c9fded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9d1e00e6963452cbcfe81dc53c1c3a2', 'request_id': 'req-68e5e632-8b41-4460-b649-808fc8c9fded'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.770515 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-68e5e632-8b41-4460-b649-808fc8c9fded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9d1e00e6963452cbcfe81dc53c1c3a2', 'request_id': 'req-68e5e632-8b41-4460-b649-808fc8c9fded'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.773548 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a417533-adfb-4a29-92bf-658a459edef9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:22.775135 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a417533-adfb-4a29-92bf-658a459edef9 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:22.783344 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-68e5e632-8b41-4460-b649-808fc8c9fded admin admin] The token cache is being invalidate because project f9d1e00e6963452cbcfe81dc53c1c3a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:22.783632 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-68e5e632-8b41-4460-b649-808fc8c9fded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.784316 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-68e5e632-8b41-4460-b649-808fc8c9fded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.785041 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1368/2738] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:22.786047 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a417533-adfb-4a29-92bf-658a459edef9 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:22.786175 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1371/2739] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:22 2022] DELETE /identity/v3/projects/f9d1e00e6963452cbcfe81dc53c1c3a2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:22.791805 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd91f0e-f121-45ca-9409-506356994366 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.792504 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd91f0e-f121-45ca-9409-506356994366 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.792659 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd91f0e-f121-45ca-9409-506356994366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.799335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a417533-adfb-4a29-92bf-658a459edef9 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:22.809192 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:22.822185 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-cbd91f0e-f121-45ca-9409-506356994366 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:22.824370 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-cbd91f0e-f121-45ca-9409-506356994366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:22.854826 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:22.863434 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90143148-1446-4c1e-b767-9e0dcb724108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:22.864049 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90143148-1446-4c1e-b767-9e0dcb724108 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.864372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90143148-1446-4c1e-b767-9e0dcb724108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.864645 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90143148-1446-4c1e-b767-9e0dcb724108 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.865294 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90143148-1446-4c1e-b767-9e0dcb724108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:22.866875 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90143148-1446-4c1e-b767-9e0dcb724108 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:22.877262 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1369/2740] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:22.877898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90143148-1446-4c1e-b767-9e0dcb724108 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:22.879539 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-90143148-1446-4c1e-b767-9e0dcb724108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:22.888019 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1372/2741] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:22.889182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd91f0e-f121-45ca-9409-506356994366 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:22.890653 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-cbd91f0e-f121-45ca-9409-506356994366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:22.898126 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90143148-1446-4c1e-b767-9e0dcb724108 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:22.902660 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd91f0e-f121-45ca-9409-506356994366 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:22.903383 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c91e4-6853-4c95-aab7-c4acab048df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:22.904029 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c91e4-6853-4c95-aab7-c4acab048df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.904326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c91e4-6853-4c95-aab7-c4acab048df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.904599 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c91e4-6853-4c95-aab7-c4acab048df3 admin admin] PATH_INFO: `/v3/users/bf3d32b7b49945cea550f13849bf9995` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.905707 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c91e4-6853-4c95-aab7-c4acab048df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf3d32b7b49945cea550f13849bf9995)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:22.907747 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d96124-d26c-4c32-9c2f-030170893cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:22.908214 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c91e4-6853-4c95-aab7-c4acab048df3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:22.908365 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d96124-d26c-4c32-9c2f-030170893cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.908641 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d96124-d26c-4c32-9c2f-030170893cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.908908 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d96124-d26c-4c32-9c2f-030170893cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.909543 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d96124-d26c-4c32-9c2f-030170893cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:22.911227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d96124-d26c-4c32-9c2f-030170893cc5 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:22.921759 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1373/2742] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:22.923919 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797b8e9-8bca-4edb-b4ba-79aac2faf47f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.924231 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797b8e9-8bca-4edb-b4ba-79aac2faf47f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.924499 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797b8e9-8bca-4edb-b4ba-79aac2faf47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.944846 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b797b8e9-8bca-4edb-b4ba-79aac2faf47f None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:22.946178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b797b8e9-8bca-4edb-b4ba-79aac2faf47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:22.954367 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1374/2743] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:22.957330 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ac6c91e4-6853-4c95-aab7-c4acab048df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf3d32b7b49945cea550f13849bf9995'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.957992 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ac6c91e4-6853-4c95-aab7-c4acab048df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf3d32b7b49945cea550f13849bf9995'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.958226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797b8e9-8bca-4edb-b4ba-79aac2faf47f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:22.965554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ac6c91e4-6853-4c95-aab7-c4acab048df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf3d32b7b49945cea550f13849bf9995'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.968551 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797b8e9-8bca-4edb-b4ba-79aac2faf47f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:22.974606 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ac6c91e4-6853-4c95-aab7-c4acab048df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf3d32b7b49945cea550f13849bf9995'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.977566 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403c039-de14-4bd3-aaa1-8c52cd01e4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:22.978231 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403c039-de14-4bd3-aaa1-8c52cd01e4f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.978575 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403c039-de14-4bd3-aaa1-8c52cd01e4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.978862 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403c039-de14-4bd3-aaa1-8c52cd01e4f4 admin admin] PATH_INFO: `/v3/projects/cffa668fa8964c5f9eda68472e7d1247/users/4a28aa09ea6349b2b75d090d5539f334/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.989224 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ac6c91e4-6853-4c95-aab7-c4acab048df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf3d32b7b49945cea550f13849bf9995'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:22.994634 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e403c039-de14-4bd3-aaa1-8c52cd01e4f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cffa668fa8964c5f9eda68472e7d1247, user_id=4a28aa09ea6349b2b75d090d5539f334, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:22.996724 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1370/2744] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:22 2022] DELETE /identity/v3/users/bf3d32b7b49945cea550f13849bf9995 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:22.997334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e403c039-de14-4bd3-aaa1-8c52cd01e4f4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.000727 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c91e4-6853-4c95-aab7-c4acab048df3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.002402 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ac6c91e4-6853-4c95-aab7-c4acab048df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:23.011636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1375/2745] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:22 2022] PUT /identity/v3/projects/cffa668fa8964c5f9eda68472e7d1247/users/4a28aa09ea6349b2b75d090d5539f334/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:23.017430 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d032866-c169-4067-a28f-464bfcf08e76 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.017889 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d032866-c169-4067-a28f-464bfcf08e76 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.018264 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d032866-c169-4067-a28f-464bfcf08e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.038306 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-4d032866-c169-4067-a28f-464bfcf08e76 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:23.039521 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4d032866-c169-4067-a28f-464bfcf08e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:23.068422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c91e4-6853-4c95-aab7-c4acab048df3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.103232 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1376/2746] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:23.105001 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c3651-9d5b-4e78-87fa-e0f09e355d16 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.105385 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c3651-9d5b-4e78-87fa-e0f09e355d16 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.105759 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c3651-9d5b-4e78-87fa-e0f09e355d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.127917 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9db369-526a-47b1-9285-ca50033fd8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.128553 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9db369-526a-47b1-9285-ca50033fd8dc admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.128834 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9db369-526a-47b1-9285-ca50033fd8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.129101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9db369-526a-47b1-9285-ca50033fd8dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.129741 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9db369-526a-47b1-9285-ca50033fd8dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:23.130037 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9db369-526a-47b1-9285-ca50033fd8dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.131614 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9db369-526a-47b1-9285-ca50033fd8dc admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.134512 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-b06c3651-9d5b-4e78-87fa-e0f09e355d16 None None] MFA Rules not processed for user `4a28aa09ea6349b2b75d090d5539f334`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:23.135749 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b06c3651-9d5b-4e78-87fa-e0f09e355d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:23.143371 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1371/2747] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:23.144270 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9db369-526a-47b1-9285-ca50033fd8dc admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.148062 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8c9db369-526a-47b1-9285-ca50033fd8dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:23.154176 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:23.191849 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:23.202032 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1377/2748] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:23.206653 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c3651-9d5b-4e78-87fa-e0f09e355d16 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.208409 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9db369-526a-47b1-9285-ca50033fd8dc admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.212050 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48598688-82b0-4254-9bf9-ffc7b09717d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.212676 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48598688-82b0-4254-9bf9-ffc7b09717d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.212954 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48598688-82b0-4254-9bf9-ffc7b09717d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.213219 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48598688-82b0-4254-9bf9-ffc7b09717d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.213904 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c3651-9d5b-4e78-87fa-e0f09e355d16 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.218435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28858638-2507-42b0-b1b8-2202a70dd852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.219590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28858638-2507-42b0-b1b8-2202a70dd852 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.219866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28858638-2507-42b0-b1b8-2202a70dd852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.220300 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28858638-2507-42b0-b1b8-2202a70dd852 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.220946 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28858638-2507-42b0-b1b8-2202a70dd852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.222682 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48598688-82b0-4254-9bf9-ffc7b09717d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.222834 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28858638-2507-42b0-b1b8-2202a70dd852 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.224082 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48598688-82b0-4254-9bf9-ffc7b09717d7 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.233593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1372/2749] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:23.234455 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48598688-82b0-4254-9bf9-ffc7b09717d7 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.235775 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-48598688-82b0-4254-9bf9-ffc7b09717d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:23.249460 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48598688-82b0-4254-9bf9-ffc7b09717d7 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.256447 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1378/2750] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:23 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:23.256625 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d46ca-c760-49b5-85fb-81205682977d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.256625 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d46ca-c760-49b5-85fb-81205682977d admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.256625 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d46ca-c760-49b5-85fb-81205682977d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.256625 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d46ca-c760-49b5-85fb-81205682977d admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.257021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d46ca-c760-49b5-85fb-81205682977d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.257865 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28858638-2507-42b0-b1b8-2202a70dd852 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.258924 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d46ca-c760-49b5-85fb-81205682977d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.292879 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1373/2751] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:23 2022] POST /identity/v3/projects => generated 393 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:23.297074 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d46ca-c760-49b5-85fb-81205682977d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.338089 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28858638-2507-42b0-b1b8-2202a70dd852 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.368179 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d46ca-c760-49b5-85fb-81205682977d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.373705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8462a-5d5b-4fc3-9f4d-4b3e9c498ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.373705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8462a-5d5b-4fc3-9f4d-4b3e9c498ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.374064 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8462a-5d5b-4fc3-9f4d-4b3e9c498ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.374407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8462a-5d5b-4fc3-9f4d-4b3e9c498ce6 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.375314 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8462a-5d5b-4fc3-9f4d-4b3e9c498ce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.377447 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8462a-5d5b-4fc3-9f4d-4b3e9c498ce6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.402464 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1374/2752] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:23 2022] POST /identity/v3/users => generated 303 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:08:23.406801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8462a-5d5b-4fc3-9f4d-4b3e9c498ce6 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.407942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e18ffe-39e3-40bd-ae92-88983861c1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.408665 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e18ffe-39e3-40bd-ae92-88983861c1e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.409019 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e18ffe-39e3-40bd-ae92-88983861c1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.409363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e18ffe-39e3-40bd-ae92-88983861c1e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.410277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e18ffe-39e3-40bd-ae92-88983861c1e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.412401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e18ffe-39e3-40bd-ae92-88983861c1e4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.418101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8462a-5d5b-4fc3-9f4d-4b3e9c498ce6 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.423919 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3d0ad-bb6b-4d03-99be-ea30c85eced5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.424707 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3d0ad-bb6b-4d03-99be-ea30c85eced5 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.425078 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3d0ad-bb6b-4d03-99be-ea30c85eced5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.425440 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3d0ad-bb6b-4d03-99be-ea30c85eced5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.426224 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3d0ad-bb6b-4d03-99be-ea30c85eced5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.428165 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3d0ad-bb6b-4d03-99be-ea30c85eced5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.438043 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1379/2753] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:23 2022] POST /identity/v3/users => generated 283 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:23.441937 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1375/2754] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:23.443070 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e18ffe-39e3-40bd-ae92-88983861c1e4 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.449041 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3d0ad-bb6b-4d03-99be-ea30c85eced5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.456447 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e18ffe-39e3-40bd-ae92-88983861c1e4 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.460235 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3d0ad-bb6b-4d03-99be-ea30c85eced5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.463608 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e2c41-4ac8-48ca-a878-ffa73754012f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.464751 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e2c41-4ac8-48ca-a878-ffa73754012f admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.464751 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e2c41-4ac8-48ca-a878-ffa73754012f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.464979 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca5bf6-7e6f-4456-94bc-9638adcfbc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.465138 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e2c41-4ac8-48ca-a878-ffa73754012f admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.465405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca5bf6-7e6f-4456-94bc-9638adcfbc1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.465672 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca5bf6-7e6f-4456-94bc-9638adcfbc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.465936 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca5bf6-7e6f-4456-94bc-9638adcfbc1f admin admin] PATH_INFO: `/v3/projects/ff3fb41175ba46ac9f6ba88df8d2d7ae/users/630df48cf05843fcb3b470d10a237d7e/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.466129 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e2c41-4ac8-48ca-a878-ffa73754012f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.467921 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e2c41-4ac8-48ca-a878-ffa73754012f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.482134 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca5bf6-7e6f-4456-94bc-9638adcfbc1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff3fb41175ba46ac9f6ba88df8d2d7ae, user_id=630df48cf05843fcb3b470d10a237d7e, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.482664 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1380/2755] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:23.486797 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e2c41-4ac8-48ca-a878-ffa73754012f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.488148 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca5bf6-7e6f-4456-94bc-9638adcfbc1f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.500031 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e2c41-4ac8-48ca-a878-ffa73754012f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.506890 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1376/2756] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:23 2022] PUT /identity/v3/projects/ff3fb41175ba46ac9f6ba88df8d2d7ae/users/630df48cf05843fcb3b470d10a237d7e/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:23.511042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca5bf6-7e6f-4456-94bc-9638adcfbc1f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.567166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:23.571251 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b053a-a6be-4bcc-9983-f6ea989a9972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.571876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b053a-a6be-4bcc-9983-f6ea989a9972 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.572187 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b053a-a6be-4bcc-9983-f6ea989a9972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.572459 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b053a-a6be-4bcc-9983-f6ea989a9972 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.573173 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b053a-a6be-4bcc-9983-f6ea989a9972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.575013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b053a-a6be-4bcc-9983-f6ea989a9972 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.576458 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:23.588921 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1381/2757] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:08:23.593087 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b053a-a6be-4bcc-9983-f6ea989a9972 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.593302 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca5bf6-7e6f-4456-94bc-9638adcfbc1f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.595957 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-981b053a-a6be-4bcc-9983-f6ea989a9972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:23.598762 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1c848-8d22-4cac-92a2-3215417de899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.599445 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1c848-8d22-4cac-92a2-3215417de899 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.599791 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1c848-8d22-4cac-92a2-3215417de899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.600157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1c848-8d22-4cac-92a2-3215417de899 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.600887 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1c848-8d22-4cac-92a2-3215417de899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.602550 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1c848-8d22-4cac-92a2-3215417de899 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.608789 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b053a-a6be-4bcc-9983-f6ea989a9972 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.616946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1377/2758] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:08:23.617112 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3357f78-6672-43d9-8ba2-631c9142882b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.617232 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1c848-8d22-4cac-92a2-3215417de899 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.617351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3357f78-6672-43d9-8ba2-631c9142882b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.617351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3357f78-6672-43d9-8ba2-631c9142882b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.617351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3357f78-6672-43d9-8ba2-631c9142882b admin admin] PATH_INFO: `/v3/projects/c4c91799ea4b4de1b8e0ba9229d8cc18` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.617351 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3357f78-6672-43d9-8ba2-631c9142882b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4c91799ea4b4de1b8e0ba9229d8cc18)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.618309 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3357f78-6672-43d9-8ba2-631c9142882b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.623993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1c848-8d22-4cac-92a2-3215417de899 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.629213 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e630f-c547-4fe7-acf4-0d307117566c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.629878 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e630f-c547-4fe7-acf4-0d307117566c admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.630158 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e630f-c547-4fe7-acf4-0d307117566c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.630435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e630f-c547-4fe7-acf4-0d307117566c admin admin] PATH_INFO: `/v3/projects/3b23bad413184a39904473780216f678/users/ae43d071f4e54b3b9542db71e066fa83/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.645563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e630f-c547-4fe7-acf4-0d307117566c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b23bad413184a39904473780216f678, user_id=ae43d071f4e54b3b9542db71e066fa83, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.649045 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e630f-c547-4fe7-acf4-0d307117566c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.663014 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1378/2759] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:23 2022] PUT /identity/v3/projects/3b23bad413184a39904473780216f678/users/ae43d071f4e54b3b9542db71e066fa83/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:23.663984 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e630f-c547-4fe7-acf4-0d307117566c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.666997 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a3357f78-6672-43d9-8ba2-631c9142882b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4c91799ea4b4de1b8e0ba9229d8cc18', 'request_id': 'req-a3357f78-6672-43d9-8ba2-631c9142882b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:23.675649 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a3357f78-6672-43d9-8ba2-631c9142882b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4c91799ea4b4de1b8e0ba9229d8cc18', 'request_id': 'req-a3357f78-6672-43d9-8ba2-631c9142882b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:23.681966 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a3357f78-6672-43d9-8ba2-631c9142882b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4c91799ea4b4de1b8e0ba9229d8cc18', 'request_id': 'req-a3357f78-6672-43d9-8ba2-631c9142882b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:23.694058 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a3357f78-6672-43d9-8ba2-631c9142882b admin admin] The token cache is being invalidate because project c4c91799ea4b4de1b8e0ba9229d8cc18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:23.694343 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a3357f78-6672-43d9-8ba2-631c9142882b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:23.694998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-a3357f78-6672-43d9-8ba2-631c9142882b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:23.696530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1382/2760] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:23 2022] DELETE /identity/v3/projects/c4c91799ea4b4de1b8e0ba9229d8cc18 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:23.700440 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8f8d8-b6c5-4ec8-b081-a4f098d7c51b None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.700440 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8f8d8-b6c5-4ec8-b081-a4f098d7c51b None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.700440 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8f8d8-b6c5-4ec8-b081-a4f098d7c51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.730438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-45d8f8d8-b6c5-4ec8-b081-a4f098d7c51b None None] MFA Rules not processed for user `ae43d071f4e54b3b9542db71e066fa83`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:23.731642 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-45d8f8d8-b6c5-4ec8-b081-a4f098d7c51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:23.741007 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e630f-c547-4fe7-acf4-0d307117566c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.746384 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc547c33-0eaf-4616-bded-1df4a507b228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.747059 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc547c33-0eaf-4616-bded-1df4a507b228 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.747339 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc547c33-0eaf-4616-bded-1df4a507b228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.747608 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc547c33-0eaf-4616-bded-1df4a507b228 admin admin] PATH_INFO: `/v3/projects/ff3fb41175ba46ac9f6ba88df8d2d7ae/users/630df48cf05843fcb3b470d10a237d7e/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.749555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc547c33-0eaf-4616-bded-1df4a507b228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff3fb41175ba46ac9f6ba88df8d2d7ae, user_id=630df48cf05843fcb3b470d10a237d7e, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.752238 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc547c33-0eaf-4616-bded-1df4a507b228 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.770783 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1379/2761] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:23 2022] PUT /identity/v3/projects/ff3fb41175ba46ac9f6ba88df8d2d7ae/users/630df48cf05843fcb3b470d10a237d7e/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:23.772085 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042c31a-b475-4b06-ad1c-ab6fd7955f9e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.772397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042c31a-b475-4b06-ad1c-ab6fd7955f9e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.772667 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042c31a-b475-4b06-ad1c-ab6fd7955f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.794140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9042c31a-b475-4b06-ad1c-ab6fd7955f9e None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:23.795362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9042c31a-b475-4b06-ad1c-ab6fd7955f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:23.801408 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1383/2762] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:23.805425 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd21f1-9bc6-417e-a876-78ccc2c78209 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.805711 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd21f1-9bc6-417e-a876-78ccc2c78209 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.806100 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd21f1-9bc6-417e-a876-78ccc2c78209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.835827 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-16dd21f1-9bc6-417e-a876-78ccc2c78209 None None] MFA Rules not processed for user `630df48cf05843fcb3b470d10a237d7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:23.837065 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-16dd21f1-9bc6-417e-a876-78ccc2c78209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:23.861601 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1380/2763] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:23.862905 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042c31a-b475-4b06-ad1c-ab6fd7955f9e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.864804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9042c31a-b475-4b06-ad1c-ab6fd7955f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:23.880406 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042c31a-b475-4b06-ad1c-ab6fd7955f9e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.885127 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314ba56-0c34-459b-b525-f70adb501f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.885722 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314ba56-0c34-459b-b525-f70adb501f1a admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.885995 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314ba56-0c34-459b-b525-f70adb501f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.886259 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314ba56-0c34-459b-b525-f70adb501f1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.886908 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a314ba56-0c34-459b-b525-f70adb501f1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.888817 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a314ba56-0c34-459b-b525-f70adb501f1a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.902729 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:23.916314 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1384/2764] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:23 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:23.917191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1381/2765] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:23 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:23.920850 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314ba56-0c34-459b-b525-f70adb501f1a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.921130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522e840-f82c-4b3b-a8ce-c9167a51abbe None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.921416 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522e840-f82c-4b3b-a8ce-c9167a51abbe None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.922227 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522e840-f82c-4b3b-a8ce-c9167a51abbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.942763 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-9522e840-f82c-4b3b-a8ce-c9167a51abbe None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:23.944237 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9522e840-f82c-4b3b-a8ce-c9167a51abbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:23.948957 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:23.986765 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314ba56-0c34-459b-b525-f70adb501f1a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.991336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a3b45-030e-4b2c-ae3d-fe02c4fca1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.991983 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a3b45-030e-4b2c-ae3d-fe02c4fca1ca admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.992287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a3b45-030e-4b2c-ae3d-fe02c4fca1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.992555 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a3b45-030e-4b2c-ae3d-fe02c4fca1ca admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.993173 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a3b45-030e-4b2c-ae3d-fe02c4fca1ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.995061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a3b45-030e-4b2c-ae3d-fe02c4fca1ca admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.005675 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1385/2766] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:24.006515 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522e840-f82c-4b3b-a8ce-c9167a51abbe None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.007800 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-9522e840-f82c-4b3b-a8ce-c9167a51abbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:24.018703 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1382/2767] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:23 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:24.022407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a3b45-030e-4b2c-ae3d-fe02c4fca1ca admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.031763 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a3b45-030e-4b2c-ae3d-fe02c4fca1ca admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.036376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bf1a9-c1a1-4a79-8fbd-1ecf03945eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.036965 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bf1a9-c1a1-4a79-8fbd-1ecf03945eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.037260 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bf1a9-c1a1-4a79-8fbd-1ecf03945eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.037514 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bf1a9-c1a1-4a79-8fbd-1ecf03945eb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.038153 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7bf1a9-c1a1-4a79-8fbd-1ecf03945eb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:24.038451 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7bf1a9-c1a1-4a79-8fbd-1ecf03945eb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.040015 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7bf1a9-c1a1-4a79-8fbd-1ecf03945eb1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.053640 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1383/2768] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:24.054461 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bf1a9-c1a1-4a79-8fbd-1ecf03945eb1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.068047 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522e840-f82c-4b3b-a8ce-c9167a51abbe None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.069463 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bf1a9-c1a1-4a79-8fbd-1ecf03945eb1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.071876 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa103a6-47b3-461f-a22c-409c743c2ccb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.072525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa103a6-47b3-461f-a22c-409c743c2ccb service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.072798 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa103a6-47b3-461f-a22c-409c743c2ccb service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.073072 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa103a6-47b3-461f-a22c-409c743c2ccb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.074790 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc1061-0ee2-422d-b95c-895bd8b7c603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.075372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc1061-0ee2-422d-b95c-895bd8b7c603 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.075676 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc1061-0ee2-422d-b95c-895bd8b7c603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.075942 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc1061-0ee2-422d-b95c-895bd8b7c603 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.076605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc1061-0ee2-422d-b95c-895bd8b7c603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.078326 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc1061-0ee2-422d-b95c-895bd8b7c603 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.088508 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1384/2769] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:24.093002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc1061-0ee2-422d-b95c-895bd8b7c603 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.103031 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc1061-0ee2-422d-b95c-895bd8b7c603 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.108125 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe76cb-915c-43fd-8034-4332502ff894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.108792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe76cb-915c-43fd-8034-4332502ff894 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.109669 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe76cb-915c-43fd-8034-4332502ff894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.110004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe76cb-915c-43fd-8034-4332502ff894 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.110687 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe76cb-915c-43fd-8034-4332502ff894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.112704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe76cb-915c-43fd-8034-4332502ff894 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.135379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa103a6-47b3-461f-a22c-409c743c2ccb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.137086 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa103a6-47b3-461f-a22c-409c743c2ccb service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.145011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1385/2770] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:24 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:24.145011 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe76cb-915c-43fd-8034-4332502ff894 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.147056 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1386/2771] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:24.147758 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa103a6-47b3-461f-a22c-409c743c2ccb service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.157811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa103a6-47b3-461f-a22c-409c743c2ccb service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.208491 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:24.210826 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe76cb-915c-43fd-8034-4332502ff894 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.215853 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e628ad-57ea-4d2b-9875-d0d30a7eb9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.216663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e628ad-57ea-4d2b-9875-d0d30a7eb9da admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.217016 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e628ad-57ea-4d2b-9875-d0d30a7eb9da admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.217358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e628ad-57ea-4d2b-9875-d0d30a7eb9da admin admin] PATH_INFO: `/v3/projects/fac3b52a31344b2a8fdb11ea962db419/users/29d89b270f984a3a9eaeb22b8b4faa23/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.224035 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79ff5f-ee47-4cbd-b4a2-75b293e8eb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.224664 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79ff5f-ee47-4cbd-b4a2-75b293e8eb27 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.224942 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79ff5f-ee47-4cbd-b4a2-75b293e8eb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.225212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79ff5f-ee47-4cbd-b4a2-75b293e8eb27 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.225832 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79ff5f-ee47-4cbd-b4a2-75b293e8eb27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.227753 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79ff5f-ee47-4cbd-b4a2-75b293e8eb27 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.232980 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e628ad-57ea-4d2b-9875-d0d30a7eb9da admin admin] RBAC: Authorizing `identity:create_grant(project_id=fac3b52a31344b2a8fdb11ea962db419, user_id=29d89b270f984a3a9eaeb22b8b4faa23, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.236441 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e628ad-57ea-4d2b-9875-d0d30a7eb9da admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.250516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1386/2772] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:24 2022] PUT /identity/v3/projects/fac3b52a31344b2a8fdb11ea962db419/users/29d89b270f984a3a9eaeb22b8b4faa23/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:24.251116 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e628ad-57ea-4d2b-9875-d0d30a7eb9da admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.255505 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1387/2773] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:24 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:24.255648 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-69e628ad-57ea-4d2b-9875-d0d30a7eb9da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:24.255777 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79ff5f-ee47-4cbd-b4a2-75b293e8eb27 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.265757 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79ff5f-ee47-4cbd-b4a2-75b293e8eb27 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.320748 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e628ad-57ea-4d2b-9875-d0d30a7eb9da admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.324399 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0015d0-521e-41ab-8474-b46da138d384 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.325001 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0015d0-521e-41ab-8474-b46da138d384 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.325286 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0015d0-521e-41ab-8474-b46da138d384 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.325557 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0015d0-521e-41ab-8474-b46da138d384 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.327370 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bc0015d0-521e-41ab-8474-b46da138d384 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:24.330072 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc546570-3c2f-475f-b351-0940e1dae165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.330688 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc546570-3c2f-475f-b351-0940e1dae165 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.330970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc546570-3c2f-475f-b351-0940e1dae165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.331244 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc546570-3c2f-475f-b351-0940e1dae165 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.331889 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc546570-3c2f-475f-b351-0940e1dae165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.332694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:24.333532 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc546570-3c2f-475f-b351-0940e1dae165 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.349984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1388/2774] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:08:24.349984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc546570-3c2f-475f-b351-0940e1dae165 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.357633 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc546570-3c2f-475f-b351-0940e1dae165 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.362956 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11f8bc-084b-404a-836a-676e7c0a81e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.363481 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11f8bc-084b-404a-836a-676e7c0a81e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.363763 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11f8bc-084b-404a-836a-676e7c0a81e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.364037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11f8bc-084b-404a-836a-676e7c0a81e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.364711 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11f8bc-084b-404a-836a-676e7c0a81e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.366277 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11f8bc-084b-404a-836a-676e7c0a81e9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.376707 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1389/2775] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:24.380428 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11f8bc-084b-404a-836a-676e7c0a81e9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.388891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0015d0-521e-41ab-8474-b46da138d384 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.389784 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11f8bc-084b-404a-836a-676e7c0a81e9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.390397 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0015d0-521e-41ab-8474-b46da138d384 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.394636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dafcd-69a8-4fd2-ab72-e741c39fb747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.395274 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dafcd-69a8-4fd2-ab72-e741c39fb747 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.395557 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dafcd-69a8-4fd2-ab72-e741c39fb747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.395844 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dafcd-69a8-4fd2-ab72-e741c39fb747 admin admin] PATH_INFO: `/v3/projects/fac3b52a31344b2a8fdb11ea962db419/users/29d89b270f984a3a9eaeb22b8b4faa23/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.397779 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649dafcd-69a8-4fd2-ab72-e741c39fb747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fac3b52a31344b2a8fdb11ea962db419, user_id=29d89b270f984a3a9eaeb22b8b4faa23, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.399310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1387/2776] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:24.400416 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0015d0-521e-41ab-8474-b46da138d384 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.400669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649dafcd-69a8-4fd2-ab72-e741c39fb747 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.411100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0015d0-521e-41ab-8474-b46da138d384 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.412585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1390/2777] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:24 2022] PUT /identity/v3/projects/fac3b52a31344b2a8fdb11ea962db419/users/29d89b270f984a3a9eaeb22b8b4faa23/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:24.424130 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee4832-c43b-4f1b-8aff-2ca8cb87cb83 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.424842 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee4832-c43b-4f1b-8aff-2ca8cb87cb83 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.425126 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee4832-c43b-4f1b-8aff-2ca8cb87cb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.460975 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-f6ee4832-c43b-4f1b-8aff-2ca8cb87cb83 None None] MFA Rules not processed for user `29d89b270f984a3a9eaeb22b8b4faa23`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:24.462185 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-f6ee4832-c43b-4f1b-8aff-2ca8cb87cb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:24.476126 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d34da-e27f-43d8-b6d3-994c4ed08ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.476669 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d34da-e27f-43d8-b6d3-994c4ed08ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.476947 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d34da-e27f-43d8-b6d3-994c4ed08ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.477216 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d34da-e27f-43d8-b6d3-994c4ed08ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.477860 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d34da-e27f-43d8-b6d3-994c4ed08ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.479826 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d34da-e27f-43d8-b6d3-994c4ed08ef3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.493650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1388/2778] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:24.497862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d34da-e27f-43d8-b6d3-994c4ed08ef3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.509229 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d34da-e27f-43d8-b6d3-994c4ed08ef3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.517150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7308a-3a7f-4b2f-8f81-a6e5bdc851e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.517150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7308a-3a7f-4b2f-8f81-a6e5bdc851e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.517150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7308a-3a7f-4b2f-8f81-a6e5bdc851e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.517549 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7308a-3a7f-4b2f-8f81-a6e5bdc851e9 admin admin] PATH_INFO: `/v3/projects/759df091a0664e68b9f38eec1f9c2e49/users/517a70383bcd42eba23483981dd40d89/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.534694 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a7308a-3a7f-4b2f-8f81-a6e5bdc851e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=759df091a0664e68b9f38eec1f9c2e49, user_id=517a70383bcd42eba23483981dd40d89, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.537453 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a7308a-3a7f-4b2f-8f81-a6e5bdc851e9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.548638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1391/2779] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:08:24.554871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1389/2780] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:24 2022] PUT /identity/v3/projects/759df091a0664e68b9f38eec1f9c2e49/users/517a70383bcd42eba23483981dd40d89/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:24.559860 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14af3f-234c-43b7-a5e5-e7600aec4093 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.560176 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14af3f-234c-43b7-a5e5-e7600aec4093 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.560499 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14af3f-234c-43b7-a5e5-e7600aec4093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.562295 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7308a-3a7f-4b2f-8f81-a6e5bdc851e9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.582386 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-ff14af3f-234c-43b7-a5e5-e7600aec4093 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:24.583725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ff14af3f-234c-43b7-a5e5-e7600aec4093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:24.614989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:24.628514 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7308a-3a7f-4b2f-8f81-a6e5bdc851e9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.632591 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0b92b-8e2a-40af-991d-cda8d18a2bc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.633301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0b92b-8e2a-40af-991d-cda8d18a2bc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.633582 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0b92b-8e2a-40af-991d-cda8d18a2bc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.633853 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0b92b-8e2a-40af-991d-cda8d18a2bc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.661785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1392/2781] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:24.672657 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14af3f-234c-43b7-a5e5-e7600aec4093 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.674298 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ff14af3f-234c-43b7-a5e5-e7600aec4093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:24.677726 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:24.717622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d0b92b-8e2a-40af-991d-cda8d18a2bc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.720347 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d0b92b-8e2a-40af-991d-cda8d18a2bc1 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.736802 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1390/2782] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:08:24.884810 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14af3f-234c-43b7-a5e5-e7600aec4093 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.888961 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09afb0bc-e6ca-4dfc-9da6-fe27da70153f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.889689 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09afb0bc-e6ca-4dfc-9da6-fe27da70153f service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.889991 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09afb0bc-e6ca-4dfc-9da6-fe27da70153f service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.890283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09afb0bc-e6ca-4dfc-9da6-fe27da70153f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.900551 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09afb0bc-e6ca-4dfc-9da6-fe27da70153f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.901977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09afb0bc-e6ca-4dfc-9da6-fe27da70153f service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.911765 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1393/2783] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:08:26.366816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0b92b-8e2a-40af-991d-cda8d18a2bc1 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:26.378123 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0b92b-8e2a-40af-991d-cda8d18a2bc1 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:26.385921 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59536bee-c918-40fb-9362-c4d314800d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:26.386571 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59536bee-c918-40fb-9362-c4d314800d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:26.386858 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59536bee-c918-40fb-9362-c4d314800d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:26.387129 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59536bee-c918-40fb-9362-c4d314800d65 admin admin] PATH_INFO: `/v3/users/ae43d071f4e54b3b9542db71e066fa83` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:26.390294 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59536bee-c918-40fb-9362-c4d314800d65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae43d071f4e54b3b9542db71e066fa83)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:26.392925 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59536bee-c918-40fb-9362-c4d314800d65 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:26.466869 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-59536bee-c918-40fb-9362-c4d314800d65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae43d071f4e54b3b9542db71e066fa83'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:26.467277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-59536bee-c918-40fb-9362-c4d314800d65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae43d071f4e54b3b9542db71e066fa83'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:26.480649 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-59536bee-c918-40fb-9362-c4d314800d65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae43d071f4e54b3b9542db71e066fa83'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:26.495120 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-59536bee-c918-40fb-9362-c4d314800d65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae43d071f4e54b3b9542db71e066fa83'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:26.511398 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-59536bee-c918-40fb-9362-c4d314800d65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae43d071f4e54b3b9542db71e066fa83'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:26.520643 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1391/2784] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:26 2022] DELETE /identity/v3/users/ae43d071f4e54b3b9542db71e066fa83 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:26.525296 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae332f5e-77c7-48fb-b361-cae20bceffd4 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:26.525804 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae332f5e-77c7-48fb-b361-cae20bceffd4 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:26.526410 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae332f5e-77c7-48fb-b361-cae20bceffd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:26.552761 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-ae332f5e-77c7-48fb-b361-cae20bceffd4 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:26.554278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ae332f5e-77c7-48fb-b361-cae20bceffd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:26.662635 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1394/2785] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:26.672843 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59536bee-c918-40fb-9362-c4d314800d65 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:26.674472 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-59536bee-c918-40fb-9362-c4d314800d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:26.745301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59536bee-c918-40fb-9362-c4d314800d65 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:26.750277 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a08848-502d-4f88-8126-a267887a3d9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:26.750570 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a08848-502d-4f88-8126-a267887a3d9a service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:26.750788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a08848-502d-4f88-8126-a267887a3d9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:26.751094 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a08848-502d-4f88-8126-a267887a3d9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:26.767742 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a08848-502d-4f88-8126-a267887a3d9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:26.769405 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a08848-502d-4f88-8126-a267887a3d9a service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:26.781892 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1392/2786] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:27.261780 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae332f5e-77c7-48fb-b361-cae20bceffd4 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:27.263328 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-ae332f5e-77c7-48fb-b361-cae20bceffd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:27.295530 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae332f5e-77c7-48fb-b361-cae20bceffd4 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:27.302347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3479b0f-1543-4ff4-93a4-df5d9e42eef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:27.303086 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3479b0f-1543-4ff4-93a4-df5d9e42eef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:27.303445 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3479b0f-1543-4ff4-93a4-df5d9e42eef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:27.303836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3479b0f-1543-4ff4-93a4-df5d9e42eef4 admin admin] PATH_INFO: `/v3/projects/3b23bad413184a39904473780216f678` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:27.306278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3479b0f-1543-4ff4-93a4-df5d9e42eef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b23bad413184a39904473780216f678)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:27.308418 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3479b0f-1543-4ff4-93a4-df5d9e42eef4 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:27.366527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e3479b0f-1543-4ff4-93a4-df5d9e42eef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b23bad413184a39904473780216f678', 'request_id': 'req-e3479b0f-1543-4ff4-93a4-df5d9e42eef4'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:27.377249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e3479b0f-1543-4ff4-93a4-df5d9e42eef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b23bad413184a39904473780216f678', 'request_id': 'req-e3479b0f-1543-4ff4-93a4-df5d9e42eef4'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:27.385508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e3479b0f-1543-4ff4-93a4-df5d9e42eef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b23bad413184a39904473780216f678', 'request_id': 'req-e3479b0f-1543-4ff4-93a4-df5d9e42eef4'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:27.395591 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:27.401313 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e3479b0f-1543-4ff4-93a4-df5d9e42eef4 admin admin] The token cache is being invalidate because project 3b23bad413184a39904473780216f678 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:27.401818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e3479b0f-1543-4ff4-93a4-df5d9e42eef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:27.402967 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e3479b0f-1543-4ff4-93a4-df5d9e42eef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:27.408508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1395/2787] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:27 2022] DELETE /identity/v3/projects/3b23bad413184a39904473780216f678 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:27.409968 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a08848-502d-4f88-8126-a267887a3d9a service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:27.425173 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a08848-502d-4f88-8126-a267887a3d9a service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:27.492453 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:27.527811 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6559b7-9cb3-456b-9ac0-2494b26c22f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:27.528721 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6559b7-9cb3-456b-9ac0-2494b26c22f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:27.530852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6559b7-9cb3-456b-9ac0-2494b26c22f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:27.531223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6559b7-9cb3-456b-9ac0-2494b26c22f1 admin admin] PATH_INFO: `/v3/users/29d89b270f984a3a9eaeb22b8b4faa23` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:27.532909 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6559b7-9cb3-456b-9ac0-2494b26c22f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29d89b270f984a3a9eaeb22b8b4faa23)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:27.538685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6559b7-9cb3-456b-9ac0-2494b26c22f1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:27.613384 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-df6559b7-9cb3-456b-9ac0-2494b26c22f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29d89b270f984a3a9eaeb22b8b4faa23'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:27.614282 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-df6559b7-9cb3-456b-9ac0-2494b26c22f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29d89b270f984a3a9eaeb22b8b4faa23'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:27.626142 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-df6559b7-9cb3-456b-9ac0-2494b26c22f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29d89b270f984a3a9eaeb22b8b4faa23'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:27.636767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-df6559b7-9cb3-456b-9ac0-2494b26c22f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29d89b270f984a3a9eaeb22b8b4faa23'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:27.652556 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-df6559b7-9cb3-456b-9ac0-2494b26c22f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29d89b270f984a3a9eaeb22b8b4faa23'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:27.662070 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1393/2788] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:27 2022] DELETE /identity/v3/users/29d89b270f984a3a9eaeb22b8b4faa23 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:28.166449 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3479b0f-1543-4ff4-93a4-df5d9e42eef4 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:28.167984 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-e3479b0f-1543-4ff4-93a4-df5d9e42eef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:28.182914 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3479b0f-1543-4ff4-93a4-df5d9e42eef4 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:28.272242 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccee5a0-5a85-4a87-9732-288bb94d9783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:28.273091 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccee5a0-5a85-4a87-9732-288bb94d9783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:28.273466 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccee5a0-5a85-4a87-9732-288bb94d9783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:28.273790 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccee5a0-5a85-4a87-9732-288bb94d9783 admin admin] PATH_INFO: `/v3/users/4a28aa09ea6349b2b75d090d5539f334` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:28.275264 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccee5a0-5a85-4a87-9732-288bb94d9783 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a28aa09ea6349b2b75d090d5539f334)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:28.277482 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccee5a0-5a85-4a87-9732-288bb94d9783 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:28.397228 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-dccee5a0-5a85-4a87-9732-288bb94d9783 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a28aa09ea6349b2b75d090d5539f334'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:28.414658 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-dccee5a0-5a85-4a87-9732-288bb94d9783 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a28aa09ea6349b2b75d090d5539f334'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:28.435983 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-dccee5a0-5a85-4a87-9732-288bb94d9783 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a28aa09ea6349b2b75d090d5539f334'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:28.449081 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-dccee5a0-5a85-4a87-9732-288bb94d9783 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a28aa09ea6349b2b75d090d5539f334'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:28.451341 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-dccee5a0-5a85-4a87-9732-288bb94d9783 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a28aa09ea6349b2b75d090d5539f334'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:28.463953 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1396/2789] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:28 2022] DELETE /identity/v3/users/4a28aa09ea6349b2b75d090d5539f334 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:28.469356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae26b9e-79e4-43fd-a576-298df5644341 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:28.469841 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae26b9e-79e4-43fd-a576-298df5644341 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:28.469938 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae26b9e-79e4-43fd-a576-298df5644341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:28.493862 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-aae26b9e-79e4-43fd-a576-298df5644341 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:28.495030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-aae26b9e-79e4-43fd-a576-298df5644341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:28.531723 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccee5a0-5a85-4a87-9732-288bb94d9783 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:28.534055 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-dccee5a0-5a85-4a87-9732-288bb94d9783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:28.619221 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1394/2790] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:08:28.636574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae26b9e-79e4-43fd-a576-298df5644341 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:28.638972 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-aae26b9e-79e4-43fd-a576-298df5644341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:28.694392 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccee5a0-5a85-4a87-9732-288bb94d9783 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:28.705833 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae26b9e-79e4-43fd-a576-298df5644341 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:28.707567 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4d3ed-0ca2-4b99-8589-980775f2d95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:28.708330 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4d3ed-0ca2-4b99-8589-980775f2d95c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:28.708527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4d3ed-0ca2-4b99-8589-980775f2d95c admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:28.708788 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4d3ed-0ca2-4b99-8589-980775f2d95c admin admin] PATH_INFO: `/v3/projects/fac3b52a31344b2a8fdb11ea962db419` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:28.710058 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4d3ed-0ca2-4b99-8589-980775f2d95c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fac3b52a31344b2a8fdb11ea962db419)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:28.712294 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4d3ed-0ca2-4b99-8589-980775f2d95c admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:28.722029 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0649aebd-0f4b-433a-ab3f-b3304ba1ac12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:28.722664 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649aebd-0f4b-433a-ab3f-b3304ba1ac12 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:28.722941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649aebd-0f4b-433a-ab3f-b3304ba1ac12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:28.723206 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649aebd-0f4b-433a-ab3f-b3304ba1ac12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:28.741627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0649aebd-0f4b-433a-ab3f-b3304ba1ac12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:28.741825 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0649aebd-0f4b-433a-ab3f-b3304ba1ac12 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:28.753295 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:28.759166 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1395/2791] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:08:28.789335 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fef4d3ed-0ca2-4b99-8589-980775f2d95c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fac3b52a31344b2a8fdb11ea962db419', 'request_id': 'req-fef4d3ed-0ca2-4b99-8589-980775f2d95c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:28.799687 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fef4d3ed-0ca2-4b99-8589-980775f2d95c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fac3b52a31344b2a8fdb11ea962db419', 'request_id': 'req-fef4d3ed-0ca2-4b99-8589-980775f2d95c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:28.807915 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fef4d3ed-0ca2-4b99-8589-980775f2d95c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fac3b52a31344b2a8fdb11ea962db419', 'request_id': 'req-fef4d3ed-0ca2-4b99-8589-980775f2d95c'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:28.824636 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fef4d3ed-0ca2-4b99-8589-980775f2d95c admin admin] The token cache is being invalidate because project fac3b52a31344b2a8fdb11ea962db419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:28.825203 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fef4d3ed-0ca2-4b99-8589-980775f2d95c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:28.826255 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-fef4d3ed-0ca2-4b99-8589-980775f2d95c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:28.829201 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1397/2792] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:28 2022] DELETE /identity/v3/projects/fac3b52a31344b2a8fdb11ea962db419 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:28.834020 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e7d8a-7716-4baf-9af4-27f27c35b0cc None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:28.834527 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e7d8a-7716-4baf-9af4-27f27c35b0cc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:28.834797 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e7d8a-7716-4baf-9af4-27f27c35b0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:28.845293 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:28.864790 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1f0e7d8a-7716-4baf-9af4-27f27c35b0cc None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:28.866435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1f0e7d8a-7716-4baf-9af4-27f27c35b0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:28.957212 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1396/2793] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:28.966281 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7653432-414d-4f15-bf94-14b6402a4223 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:28.966570 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7653432-414d-4f15-bf94-14b6402a4223 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:28.966845 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7653432-414d-4f15-bf94-14b6402a4223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:29.012325 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d7653432-414d-4f15-bf94-14b6402a4223 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:29.013565 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d7653432-414d-4f15-bf94-14b6402a4223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:29.027741 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1398/2794] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:29.033477 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e7d8a-7716-4baf-9af4-27f27c35b0cc None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:29.046422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e7d8a-7716-4baf-9af4-27f27c35b0cc None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:29.055025 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490812d-9839-4c2b-9e08-87d460fd69b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:29.055943 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490812d-9839-4c2b-9e08-87d460fd69b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:29.058217 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490812d-9839-4c2b-9e08-87d460fd69b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:29.059090 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490812d-9839-4c2b-9e08-87d460fd69b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:29.060074 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490812d-9839-4c2b-9e08-87d460fd69b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:29.060408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490812d-9839-4c2b-9e08-87d460fd69b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:29.062376 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490812d-9839-4c2b-9e08-87d460fd69b0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:29.084877 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1397/2795] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:29.086523 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7653432-414d-4f15-bf94-14b6402a4223 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:29.102364 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7653432-414d-4f15-bf94-14b6402a4223 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:29.110036 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4956334-5168-40dc-9bf2-bb8566afbbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:29.110784 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4956334-5168-40dc-9bf2-bb8566afbbfc admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:29.111682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4956334-5168-40dc-9bf2-bb8566afbbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:29.117528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4956334-5168-40dc-9bf2-bb8566afbbfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:29.118363 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4956334-5168-40dc-9bf2-bb8566afbbfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:29.120989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4956334-5168-40dc-9bf2-bb8566afbbfc admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:29.165109 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1399/2796] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:29 2022] POST /identity/v3/projects => generated 383 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:29.169364 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490812d-9839-4c2b-9e08-87d460fd69b0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:29.186602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4956334-5168-40dc-9bf2-bb8566afbbfc admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:29.192108 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490812d-9839-4c2b-9e08-87d460fd69b0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:29.195132 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d4956334-5168-40dc-9bf2-bb8566afbbfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:29.339494 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00469ca0-20b3-4453-937d-ee10acd8de8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:29.340621 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00469ca0-20b3-4453-937d-ee10acd8de8a admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:29.341146 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00469ca0-20b3-4453-937d-ee10acd8de8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:29.341630 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00469ca0-20b3-4453-937d-ee10acd8de8a admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:29.343099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00469ca0-20b3-4453-937d-ee10acd8de8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:29.346788 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00469ca0-20b3-4453-937d-ee10acd8de8a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:29.351513 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4956334-5168-40dc-9bf2-bb8566afbbfc admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:29.369059 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669faf84-f9ea-47bc-8534-72da125351f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:29.369920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669faf84-f9ea-47bc-8534-72da125351f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:29.370226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669faf84-f9ea-47bc-8534-72da125351f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:29.370501 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669faf84-f9ea-47bc-8534-72da125351f9 admin admin] PATH_INFO: `/v3/projects/cffa668fa8964c5f9eda68472e7d1247` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:29.371963 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669faf84-f9ea-47bc-8534-72da125351f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cffa668fa8964c5f9eda68472e7d1247)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:29.379468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669faf84-f9ea-47bc-8534-72da125351f9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:29.394806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1398/2797] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:29 2022] POST /identity/v3/users => generated 292 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:29.395940 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00469ca0-20b3-4453-937d-ee10acd8de8a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:29.410185 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00469ca0-20b3-4453-937d-ee10acd8de8a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:29.417566 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f4f53-dcdc-4d1c-bc8d-06176616fa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:29.419017 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f4f53-dcdc-4d1c-bc8d-06176616fa1e admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:29.419526 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f4f53-dcdc-4d1c-bc8d-06176616fa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:29.420552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f4f53-dcdc-4d1c-bc8d-06176616fa1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:29.421918 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f4f53-dcdc-4d1c-bc8d-06176616fa1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:29.424066 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f4f53-dcdc-4d1c-bc8d-06176616fa1e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:29.438554 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1399/2798] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:29.438890 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-669faf84-f9ea-47bc-8534-72da125351f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cffa668fa8964c5f9eda68472e7d1247', 'request_id': 'req-669faf84-f9ea-47bc-8534-72da125351f9'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:29.442227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f4f53-dcdc-4d1c-bc8d-06176616fa1e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:29.448410 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-669faf84-f9ea-47bc-8534-72da125351f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cffa668fa8964c5f9eda68472e7d1247', 'request_id': 'req-669faf84-f9ea-47bc-8534-72da125351f9'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:29.453521 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f4f53-dcdc-4d1c-bc8d-06176616fa1e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:29.458072 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-669faf84-f9ea-47bc-8534-72da125351f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cffa668fa8964c5f9eda68472e7d1247', 'request_id': 'req-669faf84-f9ea-47bc-8534-72da125351f9'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:29.474443 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-669faf84-f9ea-47bc-8534-72da125351f9 admin admin] The token cache is being invalidate because project cffa668fa8964c5f9eda68472e7d1247 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:29.475033 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-669faf84-f9ea-47bc-8534-72da125351f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:29.475820 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-669faf84-f9ea-47bc-8534-72da125351f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:29.477742 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1400/2799] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:29 2022] DELETE /identity/v3/projects/cffa668fa8964c5f9eda68472e7d1247 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:29.481823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669faf84-f9ea-47bc-8534-72da125351f9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:29.483643 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-669faf84-f9ea-47bc-8534-72da125351f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:29.495040 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:29.523978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48363568-bb1d-4a85-a3b4-b23e4cb35c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:29.524730 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48363568-bb1d-4a85-a3b4-b23e4cb35c8a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:29.525074 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48363568-bb1d-4a85-a3b4-b23e4cb35c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:29.525570 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48363568-bb1d-4a85-a3b4-b23e4cb35c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:29.526302 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48363568-bb1d-4a85-a3b4-b23e4cb35c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:29.528185 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48363568-bb1d-4a85-a3b4-b23e4cb35c8a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:29.542089 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1400/2800] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:08:29.547904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48363568-bb1d-4a85-a3b4-b23e4cb35c8a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:29.549880 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-48363568-bb1d-4a85-a3b4-b23e4cb35c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:29.563021 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669faf84-f9ea-47bc-8534-72da125351f9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:29.566521 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48363568-bb1d-4a85-a3b4-b23e4cb35c8a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:29.568002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed1251-fb16-4ad4-a60c-b132659eb1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:29.568893 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed1251-fb16-4ad4-a60c-b132659eb1bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:29.569146 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed1251-fb16-4ad4-a60c-b132659eb1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:29.570054 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed1251-fb16-4ad4-a60c-b132659eb1bb admin admin] PATH_INFO: `/v3/users/630df48cf05843fcb3b470d10a237d7e` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:29.571721 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed1251-fb16-4ad4-a60c-b132659eb1bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=630df48cf05843fcb3b470d10a237d7e)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:29.572524 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3899a4a-e307-4166-8ddd-74736a64a079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:29.573512 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3899a4a-e307-4166-8ddd-74736a64a079 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:29.573646 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed1251-fb16-4ad4-a60c-b132659eb1bb admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:29.573864 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3899a4a-e307-4166-8ddd-74736a64a079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:29.574342 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3899a4a-e307-4166-8ddd-74736a64a079 admin admin] PATH_INFO: `/v3/projects/039a312c4ece4bcea71d2383cb1a9ed2/users/629990e1d73342b98a377a88f0009d5b/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:29.578325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:29.595705 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3899a4a-e307-4166-8ddd-74736a64a079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=039a312c4ece4bcea71d2383cb1a9ed2, user_id=629990e1d73342b98a377a88f0009d5b, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:29.598721 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3899a4a-e307-4166-8ddd-74736a64a079 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:29.617948 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1401/2801] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:29 2022] PUT /identity/v3/projects/039a312c4ece4bcea71d2383cb1a9ed2/users/629990e1d73342b98a377a88f0009d5b/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:29.631212 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd52fd-5876-42d6-8baf-ea066a3afe44 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:29.634223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd52fd-5876-42d6-8baf-ea066a3afe44 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:29.634538 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd52fd-5876-42d6-8baf-ea066a3afe44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:29.653125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-32ed1251-fb16-4ad4-a60c-b132659eb1bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '630df48cf05843fcb3b470d10a237d7e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:29.665446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-32ed1251-fb16-4ad4-a60c-b132659eb1bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '630df48cf05843fcb3b470d10a237d7e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:29.673940 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e1fd52fd-5876-42d6-8baf-ea066a3afe44 None None] MFA Rules not processed for user `629990e1d73342b98a377a88f0009d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:29.675723 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e1fd52fd-5876-42d6-8baf-ea066a3afe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:29.688619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-32ed1251-fb16-4ad4-a60c-b132659eb1bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '630df48cf05843fcb3b470d10a237d7e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:29.700775 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-32ed1251-fb16-4ad4-a60c-b132659eb1bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '630df48cf05843fcb3b470d10a237d7e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:29.702614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-32ed1251-fb16-4ad4-a60c-b132659eb1bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '630df48cf05843fcb3b470d10a237d7e'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:29.710208 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1401/2802] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:29 2022] DELETE /identity/v3/users/630df48cf05843fcb3b470d10a237d7e => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:29.784152 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1402/2803] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:29.797154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed1251-fb16-4ad4-a60c-b132659eb1bb admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:29.799653 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-32ed1251-fb16-4ad4-a60c-b132659eb1bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:29.927325 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed1251-fb16-4ad4-a60c-b132659eb1bb admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:29.946474 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfb1e5-3e66-4ead-8d7a-992093f3335a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:29.949031 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfb1e5-3e66-4ead-8d7a-992093f3335a service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:29.950279 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfb1e5-3e66-4ead-8d7a-992093f3335a service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:29.950925 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfb1e5-3e66-4ead-8d7a-992093f3335a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.024194 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:30.049767 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dfb1e5-3e66-4ead-8d7a-992093f3335a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.051209 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dfb1e5-3e66-4ead-8d7a-992093f3335a service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:30.062673 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1402/2804] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:30.089042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8a7a4-1c6d-4d0e-a50f-28f5098fc915 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.089508 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8a7a4-1c6d-4d0e-a50f-28f5098fc915 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.089931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8a7a4-1c6d-4d0e-a50f-28f5098fc915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.093680 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfb1e5-3e66-4ead-8d7a-992093f3335a service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:30.095275 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-86dfb1e5-3e66-4ead-8d7a-992093f3335a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:30.128570 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-75f8a7a4-1c6d-4d0e-a50f-28f5098fc915 None None] MFA Rules not processed for user `517a70383bcd42eba23483981dd40d89`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:30.130346 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-75f8a7a4-1c6d-4d0e-a50f-28f5098fc915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:30.187337 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfb1e5-3e66-4ead-8d7a-992093f3335a service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.194088 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb37515-186f-4af8-a398-d61f1bd9670e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.194751 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb37515-186f-4af8-a398-d61f1bd9670e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.195048 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb37515-186f-4af8-a398-d61f1bd9670e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.195262 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb37515-186f-4af8-a398-d61f1bd9670e admin admin] PATH_INFO: `/v3/users/629990e1d73342b98a377a88f0009d5b` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.196712 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb37515-186f-4af8-a398-d61f1bd9670e admin admin] RBAC: Authorizing `identity:delete_user(user_id=629990e1d73342b98a377a88f0009d5b)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.198700 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb37515-186f-4af8-a398-d61f1bd9670e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:30.236981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1403/2805] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:08:30.238063 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8a7a4-1c6d-4d0e-a50f-28f5098fc915 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:30.239952 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-75f8a7a4-1c6d-4d0e-a50f-28f5098fc915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:30.261348 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6eb37515-186f-4af8-a398-d61f1bd9670e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '629990e1d73342b98a377a88f0009d5b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:30.262932 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8a7a4-1c6d-4d0e-a50f-28f5098fc915 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.269359 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637ffaa-dcc2-46b6-be2f-f94481c36636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.270305 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637ffaa-dcc2-46b6-be2f-f94481c36636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.270681 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637ffaa-dcc2-46b6-be2f-f94481c36636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.270889 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637ffaa-dcc2-46b6-be2f-f94481c36636 admin admin] PATH_INFO: `/v3/projects/ff3fb41175ba46ac9f6ba88df8d2d7ae` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.271713 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6eb37515-186f-4af8-a398-d61f1bd9670e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '629990e1d73342b98a377a88f0009d5b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:30.272121 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637ffaa-dcc2-46b6-be2f-f94481c36636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff3fb41175ba46ac9f6ba88df8d2d7ae)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.274207 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637ffaa-dcc2-46b6-be2f-f94481c36636 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:30.288356 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6eb37515-186f-4af8-a398-d61f1bd9670e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '629990e1d73342b98a377a88f0009d5b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:30.296167 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6eb37515-186f-4af8-a398-d61f1bd9670e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '629990e1d73342b98a377a88f0009d5b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:30.297003 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-6eb37515-186f-4af8-a398-d61f1bd9670e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '629990e1d73342b98a377a88f0009d5b'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:30.304184 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1403/2806] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:30 2022] DELETE /identity/v3/users/629990e1d73342b98a377a88f0009d5b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:30.305215 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb37515-186f-4af8-a398-d61f1bd9670e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:30.306765 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6eb37515-186f-4af8-a398-d61f1bd9670e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:30.332601 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4637ffaa-dcc2-46b6-be2f-f94481c36636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff3fb41175ba46ac9f6ba88df8d2d7ae', 'request_id': 'req-4637ffaa-dcc2-46b6-be2f-f94481c36636'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:30.340570 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4637ffaa-dcc2-46b6-be2f-f94481c36636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff3fb41175ba46ac9f6ba88df8d2d7ae', 'request_id': 'req-4637ffaa-dcc2-46b6-be2f-f94481c36636'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:30.343591 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:30.351549 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4637ffaa-dcc2-46b6-be2f-f94481c36636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff3fb41175ba46ac9f6ba88df8d2d7ae', 'request_id': 'req-4637ffaa-dcc2-46b6-be2f-f94481c36636'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:30.366324 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4637ffaa-dcc2-46b6-be2f-f94481c36636 admin admin] The token cache is being invalidate because project ff3fb41175ba46ac9f6ba88df8d2d7ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:30.366616 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4637ffaa-dcc2-46b6-be2f-f94481c36636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:30.367281 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4637ffaa-dcc2-46b6-be2f-f94481c36636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:30.368824 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1404/2807] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:30 2022] DELETE /identity/v3/projects/ff3fb41175ba46ac9f6ba88df8d2d7ae => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:30.374782 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59493c9-7a98-4ff1-b468-5824832dd547 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.375197 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59493c9-7a98-4ff1-b468-5824832dd547 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.375561 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59493c9-7a98-4ff1-b468-5824832dd547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.386735 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb37515-186f-4af8-a398-d61f1bd9670e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.392537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:30.396926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f59493c9-7a98-4ff1-b468-5824832dd547 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:30.398230 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f59493c9-7a98-4ff1-b468-5824832dd547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:30.462614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429c9b2-4f0f-4e2e-a07f-4eb6325c3d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.463507 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429c9b2-4f0f-4e2e-a07f-4eb6325c3d7a admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.463951 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429c9b2-4f0f-4e2e-a07f-4eb6325c3d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.464252 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429c9b2-4f0f-4e2e-a07f-4eb6325c3d7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.464896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429c9b2-4f0f-4e2e-a07f-4eb6325c3d7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.467522 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429c9b2-4f0f-4e2e-a07f-4eb6325c3d7a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:30.487030 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1405/2808] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:30.488497 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119dd28b-dc51-4e93-ace0-dc42120df92b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.488785 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119dd28b-dc51-4e93-ace0-dc42120df92b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.489054 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119dd28b-dc51-4e93-ace0-dc42120df92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.506932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1404/2809] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:30 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:30.508229 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429c9b2-4f0f-4e2e-a07f-4eb6325c3d7a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:30.512318 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-a429c9b2-4f0f-4e2e-a07f-4eb6325c3d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:30.514563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-119dd28b-dc51-4e93-ace0-dc42120df92b None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:30.515736 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-119dd28b-dc51-4e93-ace0-dc42120df92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:30.578393 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1406/2810] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:30.579304 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119dd28b-dc51-4e93-ace0-dc42120df92b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:30.579859 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429c9b2-4f0f-4e2e-a07f-4eb6325c3d7a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.580782 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-119dd28b-dc51-4e93-ace0-dc42120df92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:30.588855 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852618c-5e7c-49f1-b0a0-2a4f5873bd27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.589469 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852618c-5e7c-49f1-b0a0-2a4f5873bd27 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.589743 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852618c-5e7c-49f1-b0a0-2a4f5873bd27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.590009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852618c-5e7c-49f1-b0a0-2a4f5873bd27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.592552 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119dd28b-dc51-4e93-ace0-dc42120df92b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.597118 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba84324-5fb7-45fa-a2b4-33ad7bfa1ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.597735 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba84324-5fb7-45fa-a2b4-33ad7bfa1ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.598002 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba84324-5fb7-45fa-a2b4-33ad7bfa1ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.598272 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba84324-5fb7-45fa-a2b4-33ad7bfa1ee5 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.598896 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba84324-5fb7-45fa-a2b4-33ad7bfa1ee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.599989 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852618c-5e7c-49f1-b0a0-2a4f5873bd27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.600902 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba84324-5fb7-45fa-a2b4-33ad7bfa1ee5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:30.601590 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852618c-5e7c-49f1-b0a0-2a4f5873bd27 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:30.613465 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1405/2811] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:30.614808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8661a38-d0f2-4200-88fb-f27ae3b64e1f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.616471 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8661a38-d0f2-4200-88fb-f27ae3b64e1f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.616760 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8661a38-d0f2-4200-88fb-f27ae3b64e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.641067 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d8661a38-d0f2-4200-88fb-f27ae3b64e1f None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:30.642320 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d8661a38-d0f2-4200-88fb-f27ae3b64e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:30.650296 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1407/2812] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:30 2022] POST /identity/v3/users => generated 299 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:30.650992 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1406/2813] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:30.655307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8661a38-d0f2-4200-88fb-f27ae3b64e1f None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:30.656510 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba84324-5fb7-45fa-a2b4-33ad7bfa1ee5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:30.666030 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8661a38-d0f2-4200-88fb-f27ae3b64e1f None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.672882 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6ec8f-6aca-4ce4-b0b3-cc816a1a64c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.673638 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6ec8f-6aca-4ce4-b0b3-cc816a1a64c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.673969 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6ec8f-6aca-4ce4-b0b3-cc816a1a64c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.674302 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6ec8f-6aca-4ce4-b0b3-cc816a1a64c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.675334 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e6ec8f-6aca-4ce4-b0b3-cc816a1a64c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:30.675641 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e6ec8f-6aca-4ce4-b0b3-cc816a1a64c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.676469 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba84324-5fb7-45fa-a2b4-33ad7bfa1ee5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.678394 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e6ec8f-6aca-4ce4-b0b3-cc816a1a64c3 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:30.684805 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909f8a4-7155-4a5b-ba44-19213f27fe8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.686575 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909f8a4-7155-4a5b-ba44-19213f27fe8e admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.686802 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909f8a4-7155-4a5b-ba44-19213f27fe8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.687280 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909f8a4-7155-4a5b-ba44-19213f27fe8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.687904 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909f8a4-7155-4a5b-ba44-19213f27fe8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.689950 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909f8a4-7155-4a5b-ba44-19213f27fe8e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:30.694718 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1407/2814] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:30.699620 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6ec8f-6aca-4ce4-b0b3-cc816a1a64c3 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:30.711777 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6ec8f-6aca-4ce4-b0b3-cc816a1a64c3 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.717849 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1408/2815] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:30.718011 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ea503-c4d3-4d5d-b6b2-ccce3c19195a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.718332 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ea503-c4d3-4d5d-b6b2-ccce3c19195a admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.718611 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ea503-c4d3-4d5d-b6b2-ccce3c19195a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.718880 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ea503-c4d3-4d5d-b6b2-ccce3c19195a admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.719313 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909f8a4-7155-4a5b-ba44-19213f27fe8e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:30.719507 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ea503-c4d3-4d5d-b6b2-ccce3c19195a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.721641 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ea503-c4d3-4d5d-b6b2-ccce3c19195a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:30.730605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909f8a4-7155-4a5b-ba44-19213f27fe8e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.736912 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bda2d-3ad2-4606-ab24-be14cf03f6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.737585 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bda2d-3ad2-4606-ab24-be14cf03f6f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.737869 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bda2d-3ad2-4606-ab24-be14cf03f6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.738200 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bda2d-3ad2-4606-ab24-be14cf03f6f4 admin admin] PATH_INFO: `/v3/projects/0ce48c20a82e4188844001db37794dde/users/ba92b7c8f75e4977a4164308dbbcbbfc/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.762028 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1408/2816] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:30 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:30.762927 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641bda2d-3ad2-4606-ab24-be14cf03f6f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce48c20a82e4188844001db37794dde, user_id=ba92b7c8f75e4977a4164308dbbcbbfc, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.766544 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641bda2d-3ad2-4606-ab24-be14cf03f6f4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:30.767902 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ea503-c4d3-4d5d-b6b2-ccce3c19195a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:30.789740 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1409/2817] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:30 2022] PUT /identity/v3/projects/0ce48c20a82e4188844001db37794dde/users/ba92b7c8f75e4977a4164308dbbcbbfc/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:30.794864 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bda2d-3ad2-4606-ab24-be14cf03f6f4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:30.851575 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:30.880804 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:30.885977 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ea503-c4d3-4d5d-b6b2-ccce3c19195a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.905232 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bda2d-3ad2-4606-ab24-be14cf03f6f4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.914731 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072d6e0-df59-4b25-8c18-6c31436ddedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.915921 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072d6e0-df59-4b25-8c18-6c31436ddedc admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.916358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072d6e0-df59-4b25-8c18-6c31436ddedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.916641 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072d6e0-df59-4b25-8c18-6c31436ddedc admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.917333 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072d6e0-df59-4b25-8c18-6c31436ddedc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.918963 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072d6e0-df59-4b25-8c18-6c31436ddedc admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:30.933325 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1410/2818] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:08:30.938195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072d6e0-df59-4b25-8c18-6c31436ddedc admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:30.950808 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072d6e0-df59-4b25-8c18-6c31436ddedc admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.956387 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e98f4-206e-437f-ae34-1ac06a93ed46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.957242 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e98f4-206e-437f-ae34-1ac06a93ed46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.957615 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e98f4-206e-437f-ae34-1ac06a93ed46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.957982 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e98f4-206e-437f-ae34-1ac06a93ed46 admin admin] PATH_INFO: `/v3/projects/0ce48c20a82e4188844001db37794dde/users/ba92b7c8f75e4977a4164308dbbcbbfc/roles/b13ffa72344c4856901b1ba003e273fd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.960131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e98f4-206e-437f-ae34-1ac06a93ed46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce48c20a82e4188844001db37794dde, user_id=ba92b7c8f75e4977a4164308dbbcbbfc, role_id=b13ffa72344c4856901b1ba003e273fd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.962992 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e98f4-206e-437f-ae34-1ac06a93ed46 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:30.981212 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc73a7-a65b-4fcf-9c5d-a02928ace1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.981967 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc73a7-a65b-4fcf-9c5d-a02928ace1cd admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.982283 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc73a7-a65b-4fcf-9c5d-a02928ace1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.983136 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc73a7-a65b-4fcf-9c5d-a02928ace1cd admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.985454 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1411/2819] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:30 2022] PUT /identity/v3/projects/0ce48c20a82e4188844001db37794dde/users/ba92b7c8f75e4977a4164308dbbcbbfc/roles/b13ffa72344c4856901b1ba003e273fd => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:30.985630 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc73a7-a65b-4fcf-9c5d-a02928ace1cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.997752 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc73a7-a65b-4fcf-9c5d-a02928ace1cd admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.006336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e98f4-206e-437f-ae34-1ac06a93ed46 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.008487 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-da2e98f4-206e-437f-ae34-1ac06a93ed46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:31.049626 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1409/2820] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:30 2022] POST /identity/v3/users => generated 288 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:31.053784 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc73a7-a65b-4fcf-9c5d-a02928ace1cd admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.085785 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc73a7-a65b-4fcf-9c5d-a02928ace1cd admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.120358 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e98f4-206e-437f-ae34-1ac06a93ed46 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.126187 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a7ab2-ded3-4288-a46f-ec0a77b928c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.126834 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a7ab2-ded3-4288-a46f-ec0a77b928c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.127108 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a7ab2-ded3-4288-a46f-ec0a77b928c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.127711 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a7ab2-ded3-4288-a46f-ec0a77b928c4 admin admin] PATH_INFO: `/v3/projects/039a312c4ece4bcea71d2383cb1a9ed2` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.129472 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a7ab2-ded3-4288-a46f-ec0a77b928c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=039a312c4ece4bcea71d2383cb1a9ed2)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.131372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a7ab2-ded3-4288-a46f-ec0a77b928c4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.163911 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f54266-aeb9-4b86-81a7-0cb763a22da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.164672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f54266-aeb9-4b86-81a7-0cb763a22da6 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.165019 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f54266-aeb9-4b86-81a7-0cb763a22da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.165357 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f54266-aeb9-4b86-81a7-0cb763a22da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.166563 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f54266-aeb9-4b86-81a7-0cb763a22da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.172034 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f54266-aeb9-4b86-81a7-0cb763a22da6 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.186520 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1410/2821] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:31.190665 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f54266-aeb9-4b86-81a7-0cb763a22da6 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.201468 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f54266-aeb9-4b86-81a7-0cb763a22da6 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.203765 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a83a7ab2-ded3-4288-a46f-ec0a77b928c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '039a312c4ece4bcea71d2383cb1a9ed2', 'request_id': 'req-a83a7ab2-ded3-4288-a46f-ec0a77b928c4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:31.220264 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a83a7ab2-ded3-4288-a46f-ec0a77b928c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '039a312c4ece4bcea71d2383cb1a9ed2', 'request_id': 'req-a83a7ab2-ded3-4288-a46f-ec0a77b928c4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:31.229568 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a83a7ab2-ded3-4288-a46f-ec0a77b928c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '039a312c4ece4bcea71d2383cb1a9ed2', 'request_id': 'req-a83a7ab2-ded3-4288-a46f-ec0a77b928c4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:31.246241 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a83a7ab2-ded3-4288-a46f-ec0a77b928c4 admin admin] The token cache is being invalidate because project 039a312c4ece4bcea71d2383cb1a9ed2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:31.247482 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a83a7ab2-ded3-4288-a46f-ec0a77b928c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:31.248426 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a83a7ab2-ded3-4288-a46f-ec0a77b928c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:31.253457 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1412/2822] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:31 2022] DELETE /identity/v3/projects/039a312c4ece4bcea71d2383cb1a9ed2 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:31.256544 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49c455-c7ea-4ddb-9bcb-1aeb16207dab None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.258074 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49c455-c7ea-4ddb-9bcb-1aeb16207dab None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.258895 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49c455-c7ea-4ddb-9bcb-1aeb16207dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.288821 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2f49c455-c7ea-4ddb-9bcb-1aeb16207dab None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:31.290254 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2f49c455-c7ea-4ddb-9bcb-1aeb16207dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:31.292004 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8630a-e15f-4a35-82d8-c0058337442a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.292986 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8630a-e15f-4a35-82d8-c0058337442a admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.293537 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8630a-e15f-4a35-82d8-c0058337442a admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.294880 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8630a-e15f-4a35-82d8-c0058337442a admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.296902 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8630a-e15f-4a35-82d8-c0058337442a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.299667 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8630a-e15f-4a35-82d8-c0058337442a admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.300332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1413/2823] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:31.308414 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d4570-7736-4027-a4be-023916999d94 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.308642 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d4570-7736-4027-a4be-023916999d94 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.309129 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d4570-7736-4027-a4be-023916999d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.319809 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1411/2824] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:08:31.322285 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8630a-e15f-4a35-82d8-c0058337442a admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.324401 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-78c8630a-e15f-4a35-82d8-c0058337442a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:31.333950 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0d1d4570-7736-4027-a4be-023916999d94 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:31.335323 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0d1d4570-7736-4027-a4be-023916999d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:31.344587 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8630a-e15f-4a35-82d8-c0058337442a admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.349025 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1414/2825] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:31.350232 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d4570-7736-4027-a4be-023916999d94 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.353832 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18978084-2866-4e17-9df2-9db78d39162b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.355178 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18978084-2866-4e17-9df2-9db78d39162b admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.355665 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18978084-2866-4e17-9df2-9db78d39162b admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.356329 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18978084-2866-4e17-9df2-9db78d39162b admin admin] PATH_INFO: `/v3/projects/8b5f697216274e20a5c8a6e7adaa7e45/users/dd61069f4c47405ea8eb7f934d981afa/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.361592 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d4570-7736-4027-a4be-023916999d94 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.367153 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65eb16-db4b-4b5a-839f-c42ce3cda561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.367866 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65eb16-db4b-4b5a-839f-c42ce3cda561 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.368271 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65eb16-db4b-4b5a-839f-c42ce3cda561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.368735 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65eb16-db4b-4b5a-839f-c42ce3cda561 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.369330 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65eb16-db4b-4b5a-839f-c42ce3cda561 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:31.369695 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65eb16-db4b-4b5a-839f-c42ce3cda561 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.371770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65eb16-db4b-4b5a-839f-c42ce3cda561 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.385315 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18978084-2866-4e17-9df2-9db78d39162b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b5f697216274e20a5c8a6e7adaa7e45, user_id=dd61069f4c47405ea8eb7f934d981afa, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.389605 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18978084-2866-4e17-9df2-9db78d39162b admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.391461 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1415/2826] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:31.392560 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65eb16-db4b-4b5a-839f-c42ce3cda561 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.404406 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:31.405233 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65eb16-db4b-4b5a-839f-c42ce3cda561 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.414400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1412/2827] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:31 2022] PUT /identity/v3/projects/8b5f697216274e20a5c8a6e7adaa7e45/users/dd61069f4c47405ea8eb7f934d981afa/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:31.426896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc98807-7610-4697-9276-7fadbb730e03 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.427597 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc98807-7610-4697-9276-7fadbb730e03 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.428060 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc98807-7610-4697-9276-7fadbb730e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.431733 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:31.497263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-bbc98807-7610-4697-9276-7fadbb730e03 None None] MFA Rules not processed for user `dd61069f4c47405ea8eb7f934d981afa`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:31.500003 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-bbc98807-7610-4697-9276-7fadbb730e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:31.500602 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45108964-33f8-43a0-94d6-7b6f2e5291c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.501234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45108964-33f8-43a0-94d6-7b6f2e5291c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.501516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45108964-33f8-43a0-94d6-7b6f2e5291c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.501791 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45108964-33f8-43a0-94d6-7b6f2e5291c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.502484 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45108964-33f8-43a0-94d6-7b6f2e5291c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.504977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45108964-33f8-43a0-94d6-7b6f2e5291c2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.539531 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1416/2828] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:31 2022] POST /identity/v3/projects => generated 361 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:31.543518 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45108964-33f8-43a0-94d6-7b6f2e5291c2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.588349 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1413/2829] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:31.603823 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc98807-7610-4697-9276-7fadbb730e03 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.606104 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-bbc98807-7610-4697-9276-7fadbb730e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:31.640327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45108964-33f8-43a0-94d6-7b6f2e5291c2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.647028 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8829adc-cf86-4d20-bd1b-e419e00e9366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.648610 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8829adc-cf86-4d20-bd1b-e419e00e9366 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.648930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8829adc-cf86-4d20-bd1b-e419e00e9366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.649328 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8829adc-cf86-4d20-bd1b-e419e00e9366 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.649981 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8829adc-cf86-4d20-bd1b-e419e00e9366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.651969 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8829adc-cf86-4d20-bd1b-e419e00e9366 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.682765 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1417/2830] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:31 2022] POST /identity/v3/users => generated 281 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:31.684844 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8829adc-cf86-4d20-bd1b-e419e00e9366 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.700240 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8829adc-cf86-4d20-bd1b-e419e00e9366 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.706197 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e1ddf-d50d-49b4-b005-52ad4cff8081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.706830 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e1ddf-d50d-49b4-b005-52ad4cff8081 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.707093 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e1ddf-d50d-49b4-b005-52ad4cff8081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.707369 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e1ddf-d50d-49b4-b005-52ad4cff8081 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.708028 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e1ddf-d50d-49b4-b005-52ad4cff8081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.709399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc98807-7610-4697-9276-7fadbb730e03 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.709658 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e1ddf-d50d-49b4-b005-52ad4cff8081 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.728157 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1418/2831] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:31.734830 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e1ddf-d50d-49b4-b005-52ad4cff8081 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.753503 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e1ddf-d50d-49b4-b005-52ad4cff8081 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.760351 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13144de5-a6f0-4352-ad70-38beaea10f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.761603 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13144de5-a6f0-4352-ad70-38beaea10f8d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.761967 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13144de5-a6f0-4352-ad70-38beaea10f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.762255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13144de5-a6f0-4352-ad70-38beaea10f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.764310 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13144de5-a6f0-4352-ad70-38beaea10f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.766248 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13144de5-a6f0-4352-ad70-38beaea10f8d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.786042 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1419/2832] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:31.790974 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13144de5-a6f0-4352-ad70-38beaea10f8d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.791702 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8f1e0-c76c-4f93-bf74-30eff7b5e364 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.793519 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8f1e0-c76c-4f93-bf74-30eff7b5e364 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.793519 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8f1e0-c76c-4f93-bf74-30eff7b5e364 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.793519 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8f1e0-c76c-4f93-bf74-30eff7b5e364 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.815068 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13144de5-a6f0-4352-ad70-38beaea10f8d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.820231 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d45cf-5289-4cf9-a836-e86dafc98316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.820968 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d45cf-5289-4cf9-a836-e86dafc98316 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.821316 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d45cf-5289-4cf9-a836-e86dafc98316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.821652 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d45cf-5289-4cf9-a836-e86dafc98316 admin admin] PATH_INFO: `/v3/projects/63df3dc60ffa4d1eabb0b62af1943fab/users/9d94341088d24bb588c35f065cb99737/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.849927 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d45cf-5289-4cf9-a836-e86dafc98316 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63df3dc60ffa4d1eabb0b62af1943fab, user_id=9d94341088d24bb588c35f065cb99737, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.853067 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d45cf-5289-4cf9-a836-e86dafc98316 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.872150 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1420/2833] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:31 2022] PUT /identity/v3/projects/63df3dc60ffa4d1eabb0b62af1943fab/users/9d94341088d24bb588c35f065cb99737/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:31.882245 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed134b27-8601-46c4-8279-4ae327313367 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.882559 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed134b27-8601-46c4-8279-4ae327313367 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.882902 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed134b27-8601-46c4-8279-4ae327313367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.884112 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8f1e0-c76c-4f93-bf74-30eff7b5e364 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.885737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8f1e0-c76c-4f93-bf74-30eff7b5e364 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.897901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:31.904944 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1414/2834] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:08:31.942596 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ed134b27-8601-46c4-8279-4ae327313367 None None] MFA Rules not processed for user `9d94341088d24bb588c35f065cb99737`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:31.944926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ed134b27-8601-46c4-8279-4ae327313367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:32.003200 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:32.047183 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1421/2835] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:32.090430 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8f1e0-c76c-4f93-bf74-30eff7b5e364 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:32.092528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8dc8f1e0-c76c-4f93-bf74-30eff7b5e364 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:32.109572 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8f1e0-c76c-4f93-bf74-30eff7b5e364 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:32.176512 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751848c-cec8-4231-bac2-524e7d977339 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:32.177278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751848c-cec8-4231-bac2-524e7d977339 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:32.177684 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751848c-cec8-4231-bac2-524e7d977339 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:32.178055 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751848c-cec8-4231-bac2-524e7d977339 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:32.189142 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751848c-cec8-4231-bac2-524e7d977339 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:32.190821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751848c-cec8-4231-bac2-524e7d977339 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:32.201736 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1415/2836] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:32.229037 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed134b27-8601-46c4-8279-4ae327313367 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:32.382988 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed134b27-8601-46c4-8279-4ae327313367 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:32.389762 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c0656-1379-46b1-8d6d-3e8c7177f7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:32.393704 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c0656-1379-46b1-8d6d-3e8c7177f7ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:32.394069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c0656-1379-46b1-8d6d-3e8c7177f7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:32.394408 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c0656-1379-46b1-8d6d-3e8c7177f7ea admin admin] PATH_INFO: `/v3/users/9d94341088d24bb588c35f065cb99737` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:32.397509 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c0656-1379-46b1-8d6d-3e8c7177f7ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d94341088d24bb588c35f065cb99737)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:32.400392 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c0656-1379-46b1-8d6d-3e8c7177f7ea admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:32.470731 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-849c0656-1379-46b1-8d6d-3e8c7177f7ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d94341088d24bb588c35f065cb99737'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:32.471426 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-849c0656-1379-46b1-8d6d-3e8c7177f7ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d94341088d24bb588c35f065cb99737'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:32.480724 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-849c0656-1379-46b1-8d6d-3e8c7177f7ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d94341088d24bb588c35f065cb99737'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:32.489480 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-849c0656-1379-46b1-8d6d-3e8c7177f7ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d94341088d24bb588c35f065cb99737'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:32.502959 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-849c0656-1379-46b1-8d6d-3e8c7177f7ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d94341088d24bb588c35f065cb99737'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:32.512442 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1422/2837] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:32 2022] DELETE /identity/v3/users/9d94341088d24bb588c35f065cb99737 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:32.514459 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bd327-97f9-449e-b5f7-7984b1cf81b2 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:32.514749 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bd327-97f9-449e-b5f7-7984b1cf81b2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:32.515022 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bd327-97f9-449e-b5f7-7984b1cf81b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:32.538210 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1f2bd327-97f9-449e-b5f7-7984b1cf81b2 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:32.539428 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1f2bd327-97f9-449e-b5f7-7984b1cf81b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:32.609438 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1416/2838] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:32.617683 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c0656-1379-46b1-8d6d-3e8c7177f7ea admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:32.619905 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-849c0656-1379-46b1-8d6d-3e8c7177f7ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:32.684505 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c0656-1379-46b1-8d6d-3e8c7177f7ea admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:32.690153 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ecbb00-ae66-4d08-ae86-b839e607f68e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:32.690718 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecbb00-ae66-4d08-ae86-b839e607f68e service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:32.691059 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecbb00-ae66-4d08-ae86-b839e607f68e service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:32.691322 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecbb00-ae66-4d08-ae86-b839e607f68e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:32.702722 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ecbb00-ae66-4d08-ae86-b839e607f68e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:32.704388 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ecbb00-ae66-4d08-ae86-b839e607f68e service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:32.713886 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:32.721598 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1423/2839] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:33.176061 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2bd327-97f9-449e-b5f7-7984b1cf81b2 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.177317 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1f2bd327-97f9-449e-b5f7-7984b1cf81b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:33.190340 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2bd327-97f9-449e-b5f7-7984b1cf81b2 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.195682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c7d84-1601-4a73-8cae-4c7e12f41843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.196799 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c7d84-1601-4a73-8cae-4c7e12f41843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.197127 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c7d84-1601-4a73-8cae-4c7e12f41843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.197386 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c7d84-1601-4a73-8cae-4c7e12f41843 admin admin] PATH_INFO: `/v3/projects/63df3dc60ffa4d1eabb0b62af1943fab` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.198513 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c7d84-1601-4a73-8cae-4c7e12f41843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63df3dc60ffa4d1eabb0b62af1943fab)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.200556 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c7d84-1601-4a73-8cae-4c7e12f41843 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.252660 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e12c7d84-1601-4a73-8cae-4c7e12f41843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63df3dc60ffa4d1eabb0b62af1943fab', 'request_id': 'req-e12c7d84-1601-4a73-8cae-4c7e12f41843'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:33.261477 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e12c7d84-1601-4a73-8cae-4c7e12f41843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63df3dc60ffa4d1eabb0b62af1943fab', 'request_id': 'req-e12c7d84-1601-4a73-8cae-4c7e12f41843'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:33.268888 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e12c7d84-1601-4a73-8cae-4c7e12f41843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63df3dc60ffa4d1eabb0b62af1943fab', 'request_id': 'req-e12c7d84-1601-4a73-8cae-4c7e12f41843'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:33.285112 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e12c7d84-1601-4a73-8cae-4c7e12f41843 admin admin] The token cache is being invalidate because project 63df3dc60ffa4d1eabb0b62af1943fab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:33.285396 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e12c7d84-1601-4a73-8cae-4c7e12f41843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:33.286146 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-e12c7d84-1601-4a73-8cae-4c7e12f41843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:33.287978 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1417/2840] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:33 2022] DELETE /identity/v3/projects/63df3dc60ffa4d1eabb0b62af1943fab => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:33.292816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0bbfd-5e74-476e-a182-9950336a9a1c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.293195 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0bbfd-5e74-476e-a182-9950336a9a1c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.293540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0bbfd-5e74-476e-a182-9950336a9a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.316125 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-72a0bbfd-5e74-476e-a182-9950336a9a1c None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:33.317871 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-72a0bbfd-5e74-476e-a182-9950336a9a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:33.385231 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1424/2841] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:33.393138 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bb22b-feba-44e7-88db-5af3f8f1315d None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.393614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bb22b-feba-44e7-88db-5af3f8f1315d None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.400716 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bb22b-feba-44e7-88db-5af3f8f1315d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.444367 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-fc8bb22b-feba-44e7-88db-5af3f8f1315d None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:33.446903 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fc8bb22b-feba-44e7-88db-5af3f8f1315d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:33.460037 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1418/2842] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:33.469754 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0bbfd-5e74-476e-a182-9950336a9a1c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.484734 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0bbfd-5e74-476e-a182-9950336a9a1c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.496391 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32439b33-2002-4e7a-9ac2-93a4a90b0eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.497499 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32439b33-2002-4e7a-9ac2-93a4a90b0eaf admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.497934 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32439b33-2002-4e7a-9ac2-93a4a90b0eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.498351 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32439b33-2002-4e7a-9ac2-93a4a90b0eaf admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.499374 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32439b33-2002-4e7a-9ac2-93a4a90b0eaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:33.499834 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32439b33-2002-4e7a-9ac2-93a4a90b0eaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.501627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32439b33-2002-4e7a-9ac2-93a4a90b0eaf admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.536516 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1425/2843] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:33.538518 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8bb22b-feba-44e7-88db-5af3f8f1315d None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.563730 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8bb22b-feba-44e7-88db-5af3f8f1315d None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.579504 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7d0d4-ce57-4d5f-a306-068bc3651291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.581816 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7d0d4-ce57-4d5f-a306-068bc3651291 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.582278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7d0d4-ce57-4d5f-a306-068bc3651291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.583445 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7d0d4-ce57-4d5f-a306-068bc3651291 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.584117 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7d0d4-ce57-4d5f-a306-068bc3651291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.589929 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7d0d4-ce57-4d5f-a306-068bc3651291 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.644225 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:33.662319 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1419/2844] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:33 2022] POST /identity/v3/projects => generated 391 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:33.666507 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32439b33-2002-4e7a-9ac2-93a4a90b0eaf admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.677724 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32439b33-2002-4e7a-9ac2-93a4a90b0eaf admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.802858 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25028e-d122-44b5-b2c0-e3e131fdd70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.803628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25028e-d122-44b5-b2c0-e3e131fdd70d admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.803976 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25028e-d122-44b5-b2c0-e3e131fdd70d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.804338 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25028e-d122-44b5-b2c0-e3e131fdd70d admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.805487 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25028e-d122-44b5-b2c0-e3e131fdd70d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.807564 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25028e-d122-44b5-b2c0-e3e131fdd70d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.840870 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7d0d4-ce57-4d5f-a306-068bc3651291 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.849067 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1426/2845] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:33 2022] POST /identity/v3/users => generated 296 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:08:33.854373 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7d0d4-ce57-4d5f-a306-068bc3651291 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.860971 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02994ceb-01a7-40cd-9d58-49dd979b6543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.862781 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02994ceb-01a7-40cd-9d58-49dd979b6543 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.863074 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02994ceb-01a7-40cd-9d58-49dd979b6543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.863400 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02994ceb-01a7-40cd-9d58-49dd979b6543 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.864215 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02994ceb-01a7-40cd-9d58-49dd979b6543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.866086 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02994ceb-01a7-40cd-9d58-49dd979b6543 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.891088 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25028e-d122-44b5-b2c0-e3e131fdd70d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.897602 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1420/2846] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:33.901745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25028e-d122-44b5-b2c0-e3e131fdd70d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.908804 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53edf3-49dd-4196-8ef4-08cf02ad6f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.909432 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53edf3-49dd-4196-8ef4-08cf02ad6f94 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.909714 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53edf3-49dd-4196-8ef4-08cf02ad6f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.909985 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53edf3-49dd-4196-8ef4-08cf02ad6f94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.910637 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53edf3-49dd-4196-8ef4-08cf02ad6f94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.912364 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53edf3-49dd-4196-8ef4-08cf02ad6f94 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.929227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1427/2847] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:08:33.933510 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02994ceb-01a7-40cd-9d58-49dd979b6543 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.969894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02994ceb-01a7-40cd-9d58-49dd979b6543 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.983927 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b85d01-aae5-4db7-99c5-3ce3208fd1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.984970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b85d01-aae5-4db7-99c5-3ce3208fd1ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.984970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b85d01-aae5-4db7-99c5-3ce3208fd1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.985268 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b85d01-aae5-4db7-99c5-3ce3208fd1ec admin admin] PATH_INFO: `/v3/projects/00fb6639cee94a29acf1e3aaff58b8e4/users/58db3d95856949138e66245e7bd0319f/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:34.004998 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b85d01-aae5-4db7-99c5-3ce3208fd1ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=00fb6639cee94a29acf1e3aaff58b8e4, user_id=58db3d95856949138e66245e7bd0319f, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:34.007767 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b85d01-aae5-4db7-99c5-3ce3208fd1ec admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:34.032527 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1421/2848] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:33 2022] PUT /identity/v3/projects/00fb6639cee94a29acf1e3aaff58b8e4/users/58db3d95856949138e66245e7bd0319f/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:34.041235 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4f31a-6ba9-423f-8c34-e7ac71f71a9b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:34.041794 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4f31a-6ba9-423f-8c34-e7ac71f71a9b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:34.042273 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4f31a-6ba9-423f-8c34-e7ac71f71a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:34.094563 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-bec4f31a-6ba9-423f-8c34-e7ac71f71a9b None None] MFA Rules not processed for user `58db3d95856949138e66245e7bd0319f`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:34.096905 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bec4f31a-6ba9-423f-8c34-e7ac71f71a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:34.218558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:34.233412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1428/2849] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:34.245190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b85d01-aae5-4db7-99c5-3ce3208fd1ec admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:34.247138 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-51b85d01-aae5-4db7-99c5-3ce3208fd1ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:34.402662 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b85d01-aae5-4db7-99c5-3ce3208fd1ec admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:34.407569 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e0ef9-8bed-4f8d-b96f-a82a50a3eca7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:34.408226 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e0ef9-8bed-4f8d-b96f-a82a50a3eca7 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:34.408505 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e0ef9-8bed-4f8d-b96f-a82a50a3eca7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:34.408774 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e0ef9-8bed-4f8d-b96f-a82a50a3eca7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:34.418567 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e0ef9-8bed-4f8d-b96f-a82a50a3eca7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:34.426523 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e0ef9-8bed-4f8d-b96f-a82a50a3eca7 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:34.446669 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1422/2850] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:35.817154 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4f31a-6ba9-423f-8c34-e7ac71f71a9b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:35.819012 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bec4f31a-6ba9-423f-8c34-e7ac71f71a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:35.928332 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4f31a-6ba9-423f-8c34-e7ac71f71a9b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:35.941306 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106a061-f33e-4ae7-b13a-c7701e3d28d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:35.942394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106a061-f33e-4ae7-b13a-c7701e3d28d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:35.942839 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106a061-f33e-4ae7-b13a-c7701e3d28d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:35.943291 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106a061-f33e-4ae7-b13a-c7701e3d28d2 admin admin] PATH_INFO: `/v3/users/dd61069f4c47405ea8eb7f934d981afa` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:35.944693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106a061-f33e-4ae7-b13a-c7701e3d28d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd61069f4c47405ea8eb7f934d981afa)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:35.946792 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106a061-f33e-4ae7-b13a-c7701e3d28d2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:36.018343 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1106a061-f33e-4ae7-b13a-c7701e3d28d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd61069f4c47405ea8eb7f934d981afa'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.019278 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1106a061-f33e-4ae7-b13a-c7701e3d28d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd61069f4c47405ea8eb7f934d981afa'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.041699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1106a061-f33e-4ae7-b13a-c7701e3d28d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd61069f4c47405ea8eb7f934d981afa'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.051643 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1106a061-f33e-4ae7-b13a-c7701e3d28d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd61069f4c47405ea8eb7f934d981afa'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.069125 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1106a061-f33e-4ae7-b13a-c7701e3d28d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd61069f4c47405ea8eb7f934d981afa'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.076222 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1429/2851] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:35 2022] DELETE /identity/v3/users/dd61069f4c47405ea8eb7f934d981afa => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:36.080721 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a8d63-0471-4984-b406-93aed0f6f01a None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:36.081200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a8d63-0471-4984-b406-93aed0f6f01a None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:36.081517 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a8d63-0471-4984-b406-93aed0f6f01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:36.104307 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-fa9a8d63-0471-4984-b406-93aed0f6f01a None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:36.105547 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fa9a8d63-0471-4984-b406-93aed0f6f01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:36.188614 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1423/2852] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:36.199680 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106a061-f33e-4ae7-b13a-c7701e3d28d2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:36.202136 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1106a061-f33e-4ae7-b13a-c7701e3d28d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:36.307716 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106a061-f33e-4ae7-b13a-c7701e3d28d2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:36.318435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508c136-1a4b-4f01-abd0-04a34d571c0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:36.319083 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508c136-1a4b-4f01-abd0-04a34d571c0f service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:36.319423 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508c136-1a4b-4f01-abd0-04a34d571c0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:36.319628 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508c136-1a4b-4f01-abd0-04a34d571c0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:36.358697 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508c136-1a4b-4f01-abd0-04a34d571c0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:36.362300 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508c136-1a4b-4f01-abd0-04a34d571c0f service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:36.386444 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1430/2853] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:36.829223 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a8d63-0471-4984-b406-93aed0f6f01a None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:36.829811 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a35ca1-e672-40c0-91e5-0b57f6b229ae None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:36.832147 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a35ca1-e672-40c0-91e5-0b57f6b229ae None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:36.832848 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a35ca1-e672-40c0-91e5-0b57f6b229ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:36.835209 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fa9a8d63-0471-4984-b406-93aed0f6f01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:36.863372 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a8d63-0471-4984-b406-93aed0f6f01a None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:36.873921 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f43a4-4ea6-4c23-ae04-53ebd57e5199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:36.875480 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f43a4-4ea6-4c23-ae04-53ebd57e5199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:36.876496 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f43a4-4ea6-4c23-ae04-53ebd57e5199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:36.876755 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f43a4-4ea6-4c23-ae04-53ebd57e5199 admin admin] PATH_INFO: `/v3/projects/8b5f697216274e20a5c8a6e7adaa7e45` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:36.877782 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b2a35ca1-e672-40c0-91e5-0b57f6b229ae None None] MFA Rules not processed for user `ba92b7c8f75e4977a4164308dbbcbbfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:36.878993 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b2a35ca1-e672-40c0-91e5-0b57f6b229ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:36.881600 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f43a4-4ea6-4c23-ae04-53ebd57e5199 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b5f697216274e20a5c8a6e7adaa7e45)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:36.883667 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f43a4-4ea6-4c23-ae04-53ebd57e5199 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:36.890713 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:36.953009 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-999f43a4-4ea6-4c23-ae04-53ebd57e5199 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b5f697216274e20a5c8a6e7adaa7e45', 'request_id': 'req-999f43a4-4ea6-4c23-ae04-53ebd57e5199'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.962184 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-999f43a4-4ea6-4c23-ae04-53ebd57e5199 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b5f697216274e20a5c8a6e7adaa7e45', 'request_id': 'req-999f43a4-4ea6-4c23-ae04-53ebd57e5199'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.966631 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:36.968833 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-999f43a4-4ea6-4c23-ae04-53ebd57e5199 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b5f697216274e20a5c8a6e7adaa7e45', 'request_id': 'req-999f43a4-4ea6-4c23-ae04-53ebd57e5199'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.979101 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1431/2854] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:36.989188 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-999f43a4-4ea6-4c23-ae04-53ebd57e5199 admin admin] The token cache is being invalidate because project 8b5f697216274e20a5c8a6e7adaa7e45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:36.989546 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-999f43a4-4ea6-4c23-ae04-53ebd57e5199 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.990278 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-999f43a4-4ea6-4c23-ae04-53ebd57e5199 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:36.992362 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1424/2855] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:36 2022] DELETE /identity/v3/projects/8b5f697216274e20a5c8a6e7adaa7e45 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:36.993119 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a35ca1-e672-40c0-91e5-0b57f6b229ae None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:36.997483 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b2a35ca1-e672-40c0-91e5-0b57f6b229ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:36.997646 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd8786-37b7-487b-818d-28450b9e635f None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:36.997851 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd8786-37b7-487b-818d-28450b9e635f None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:36.998101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd8786-37b7-487b-818d-28450b9e635f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.020064 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-8bfd8786-37b7-487b-818d-28450b9e635f None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:37.021347 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8bfd8786-37b7-487b-818d-28450b9e635f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:37.084929 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a35ca1-e672-40c0-91e5-0b57f6b229ae None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.089720 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1425/2856] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:08:37.098044 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5001bd-6499-454a-b033-c5ec20ada155 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.098323 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5001bd-6499-454a-b033-c5ec20ada155 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.098594 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5001bd-6499-454a-b033-c5ec20ada155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.124027 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-aa5001bd-6499-454a-b033-c5ec20ada155 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:37.125290 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-aa5001bd-6499-454a-b033-c5ec20ada155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:37.133619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1426/2857] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:08:37.138188 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5001bd-6499-454a-b033-c5ec20ada155 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.147677 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5001bd-6499-454a-b033-c5ec20ada155 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.152431 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fcc52b-9ecc-44aa-a09d-7e7eb0a88893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.152622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702818f-5182-49d3-8a75-9e2354dcc74e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5979387f7aa54619b8d429368c306cc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.153585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcc52b-9ecc-44aa-a09d-7e7eb0a88893 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.153865 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcc52b-9ecc-44aa-a09d-7e7eb0a88893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.154134 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcc52b-9ecc-44aa-a09d-7e7eb0a88893 admin admin] PATH_INFO: `/v3/domains` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.154257 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702818f-5182-49d3-8a75-9e2354dcc74e service glance] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.154496 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702818f-5182-49d3-8a75-9e2354dcc74e service glance] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.154836 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702818f-5182-49d3-8a75-9e2354dcc74e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.154951 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fcc52b-9ecc-44aa-a09d-7e7eb0a88893 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81476) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:37.155172 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fcc52b-9ecc-44aa-a09d-7e7eb0a88893 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.156822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fcc52b-9ecc-44aa-a09d-7e7eb0a88893 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.156946 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c702818f-5182-49d3-8a75-9e2354dcc74e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:37.171540 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1427/2858] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:37.177974 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fcc52b-9ecc-44aa-a09d-7e7eb0a88893 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.188841 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fcc52b-9ecc-44aa-a09d-7e7eb0a88893 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.194813 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8ae38-938a-413e-a9ea-d91bdc4f6bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.195436 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8ae38-938a-413e-a9ea-d91bdc4f6bb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.195718 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8ae38-938a-413e-a9ea-d91bdc4f6bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.195994 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8ae38-938a-413e-a9ea-d91bdc4f6bb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.196662 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8ae38-938a-413e-a9ea-d91bdc4f6bb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.199213 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8ae38-938a-413e-a9ea-d91bdc4f6bb9 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.218639 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:37.223542 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702818f-5182-49d3-8a75-9e2354dcc74e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.226367 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702818f-5182-49d3-8a75-9e2354dcc74e service glance] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.234324 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1428/2859] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:37 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:08:37.237263 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1432/2860] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:08:37.238510 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8ae38-938a-413e-a9ea-d91bdc4f6bb9 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.317677 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8ae38-938a-413e-a9ea-d91bdc4f6bb9 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.322771 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1569346f-6af8-46ba-b8c3-79ef34cd7df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.323498 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569346f-6af8-46ba-b8c3-79ef34cd7df8 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.323840 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569346f-6af8-46ba-b8c3-79ef34cd7df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.324223 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569346f-6af8-46ba-b8c3-79ef34cd7df8 admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.324994 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1569346f-6af8-46ba-b8c3-79ef34cd7df8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.327005 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1569346f-6af8-46ba-b8c3-79ef34cd7df8 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.354525 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1429/2861] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:37 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:08:37.357823 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702818f-5182-49d3-8a75-9e2354dcc74e service glance] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.369646 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c934cec-6fe7-48be-aea5-443a9a8ee257 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.369838 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702818f-5182-49d3-8a75-9e2354dcc74e service glance] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.370101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c934cec-6fe7-48be-aea5-443a9a8ee257 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.370449 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c934cec-6fe7-48be-aea5-443a9a8ee257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.374654 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5555a1d-39b2-4c5a-863c-b3de29d8fbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.375256 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5555a1d-39b2-4c5a-863c-b3de29d8fbd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.375534 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5555a1d-39b2-4c5a-863c-b3de29d8fbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.375873 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5555a1d-39b2-4c5a-863c-b3de29d8fbd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.376547 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5555a1d-39b2-4c5a-863c-b3de29d8fbd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.378353 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5555a1d-39b2-4c5a-863c-b3de29d8fbd6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.389682 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-8c934cec-6fe7-48be-aea5-443a9a8ee257 None None] MFA Rules not processed for user `b11abed8d4424a1e9957c3dc9aefadd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:37.390178 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1433/2862] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:37.390950 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-8c934cec-6fe7-48be-aea5-443a9a8ee257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:37.394569 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5555a1d-39b2-4c5a-863c-b3de29d8fbd6 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.405111 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5555a1d-39b2-4c5a-863c-b3de29d8fbd6 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.409731 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152d565-884b-4e61-855e-15c4ca5b7d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.410327 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152d565-884b-4e61-855e-15c4ca5b7d71 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.410608 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152d565-884b-4e61-855e-15c4ca5b7d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.410880 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152d565-884b-4e61-855e-15c4ca5b7d71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.411558 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152d565-884b-4e61-855e-15c4ca5b7d71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.413171 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152d565-884b-4e61-855e-15c4ca5b7d71 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.423787 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1434/2863] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:37.425548 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:37.427852 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152d565-884b-4e61-855e-15c4ca5b7d71 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.438225 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152d565-884b-4e61-855e-15c4ca5b7d71 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.443361 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443832e-da76-4145-b3f3-cc6bde7377df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.444046 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443832e-da76-4145-b3f3-cc6bde7377df admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.444356 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443832e-da76-4145-b3f3-cc6bde7377df admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.444627 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443832e-da76-4145-b3f3-cc6bde7377df admin admin] PATH_INFO: `/v3/projects/7c4762824948495781926df2399c75c1/users/64d2b4e534884384b0997e2fd976581f/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.452672 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1430/2864] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 24 09:08:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:08:37.460636 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443832e-da76-4145-b3f3-cc6bde7377df admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c4762824948495781926df2399c75c1, user_id=64d2b4e534884384b0997e2fd976581f, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.463768 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443832e-da76-4145-b3f3-cc6bde7377df admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.481532 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1435/2865] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:37 2022] PUT /identity/v3/projects/7c4762824948495781926df2399c75c1/users/64d2b4e534884384b0997e2fd976581f/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:37.491662 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d898e92-bc16-44bb-b5dd-bbd7695b1d69 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.491970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d898e92-bc16-44bb-b5dd-bbd7695b1d69 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.492282 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d898e92-bc16-44bb-b5dd-bbd7695b1d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.513439 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443832e-da76-4145-b3f3-cc6bde7377df admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.515144 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5443832e-da76-4145-b3f3-cc6bde7377df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:37.524890 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-1d898e92-bc16-44bb-b5dd-bbd7695b1d69 None None] MFA Rules not processed for user `64d2b4e534884384b0997e2fd976581f`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:37.526136 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1d898e92-bc16-44bb-b5dd-bbd7695b1d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:37.602573 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1431/2866] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:37.612385 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443832e-da76-4145-b3f3-cc6bde7377df admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.614811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d898e92-bc16-44bb-b5dd-bbd7695b1d69 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.616152 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1d898e92-bc16-44bb-b5dd-bbd7695b1d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:37.617597 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca78bf-92f3-4a1c-834b-73f9472b8142 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46b1417d4ac24c8982b2052f4e99042a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.618235 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca78bf-92f3-4a1c-834b-73f9472b8142 service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.618514 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca78bf-92f3-4a1c-834b-73f9472b8142 service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.618781 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca78bf-92f3-4a1c-834b-73f9472b8142 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.676808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d898e92-bc16-44bb-b5dd-bbd7695b1d69 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.681240 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5b50e-2b90-4000-ae08-ceaf8db4e263 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.681683 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ca78bf-92f3-4a1c-834b-73f9472b8142 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.681875 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5b50e-2b90-4000-ae08-ceaf8db4e263 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.682126 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5b50e-2b90-4000-ae08-ceaf8db4e263 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.682399 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5b50e-2b90-4000-ae08-ceaf8db4e263 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.683393 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ca78bf-92f3-4a1c-834b-73f9472b8142 service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.692653 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1436/2867] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 24 09:08:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 09:08:37.697185 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5b50e-2b90-4000-ae08-ceaf8db4e263 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.698836 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5b50e-2b90-4000-ae08-ceaf8db4e263 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.713013 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1432/2868] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:08:46.612834 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b725bb-0130-4f18-85c2-bae1030f45d2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:46.613394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b725bb-0130-4f18-85c2-bae1030f45d2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:46.613394 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b725bb-0130-4f18-85c2-bae1030f45d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:46.632770 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-75b725bb-0130-4f18-85c2-bae1030f45d2 None None] MFA Rules not processed for user `517a70383bcd42eba23483981dd40d89`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:46.633954 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-75b725bb-0130-4f18-85c2-bae1030f45d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:46.666087 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:46.690481 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1437/2869] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:47.592057 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5b50e-2b90-4000-ae08-ceaf8db4e263 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:47.593601 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-fee5b50e-2b90-4000-ae08-ceaf8db4e263 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:47.664073 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5b50e-2b90-4000-ae08-ceaf8db4e263 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:47.669263 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77cc642-d4f1-4ff5-af75-857288c6fc27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33a0844a1c8f4ea4932e12d53e5ee9ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:47.669884 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cc642-d4f1-4ff5-af75-857288c6fc27 service placement] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:47.670170 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cc642-d4f1-4ff5-af75-857288c6fc27 service placement] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:47.670443 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cc642-d4f1-4ff5-af75-857288c6fc27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:47.672256 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b77cc642-d4f1-4ff5-af75-857288c6fc27 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:47.684248 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77cc642-d4f1-4ff5-af75-857288c6fc27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:47.685843 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77cc642-d4f1-4ff5-af75-857288c6fc27 service placement] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:47.695640 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1433/2870] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:08:51.929572 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da2047-e67d-4a04-b741-98ded6ec3767 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:51.930176 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da2047-e67d-4a04-b741-98ded6ec3767 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:51.930298 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da2047-e67d-4a04-b741-98ded6ec3767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:51.950119 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-68da2047-e67d-4a04-b741-98ded6ec3767 None None] MFA Rules not processed for user `b11abed8d4424a1e9957c3dc9aefadd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:51.951421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-68da2047-e67d-4a04-b741-98ded6ec3767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:51.968978 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1438/2871] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 24 09:08:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:51.982386 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77cc642-d4f1-4ff5-af75-857288c6fc27 service placement] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:51.989156 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:51.994161 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77cc642-d4f1-4ff5-af75-857288c6fc27 service placement] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:52.000523 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f6b33-357f-40dc-89c6-fd6ff06e3be0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46b1417d4ac24c8982b2052f4e99042a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:52.001147 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f6b33-357f-40dc-89c6-fd6ff06e3be0 service swift] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:52.001426 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f6b33-357f-40dc-89c6-fd6ff06e3be0 service swift] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:52.001697 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f6b33-357f-40dc-89c6-fd6ff06e3be0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:52.011606 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f6b33-357f-40dc-89c6-fd6ff06e3be0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:52.013245 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f6b33-357f-40dc-89c6-fd6ff06e3be0 service swift] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:52.021751 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1434/2872] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 24 09:08:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:08:58.162744 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da2047-e67d-4a04-b741-98ded6ec3767 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:58.165058 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-68da2047-e67d-4a04-b741-98ded6ec3767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:58.356600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da2047-e67d-4a04-b741-98ded6ec3767 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:58.365745 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678b0a4-071f-463c-8611-f63e59249762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:58.366585 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678b0a4-071f-463c-8611-f63e59249762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:58.366889 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678b0a4-071f-463c-8611-f63e59249762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:58.369237 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678b0a4-071f-463c-8611-f63e59249762 admin admin] PATH_INFO: `/v3/users/58db3d95856949138e66245e7bd0319f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:58.371600 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a678b0a4-071f-463c-8611-f63e59249762 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58db3d95856949138e66245e7bd0319f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:58.375691 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a678b0a4-071f-463c-8611-f63e59249762 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:58.448717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a678b0a4-071f-463c-8611-f63e59249762 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58db3d95856949138e66245e7bd0319f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:58.449767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a678b0a4-071f-463c-8611-f63e59249762 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58db3d95856949138e66245e7bd0319f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:58.466004 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a678b0a4-071f-463c-8611-f63e59249762 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58db3d95856949138e66245e7bd0319f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:58.475787 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a678b0a4-071f-463c-8611-f63e59249762 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58db3d95856949138e66245e7bd0319f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:58.489078 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a678b0a4-071f-463c-8611-f63e59249762 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58db3d95856949138e66245e7bd0319f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:58.495352 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1439/2873] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:58 2022] DELETE /identity/v3/users/58db3d95856949138e66245e7bd0319f => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:58.500508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac63475-2c02-4ed7-9630-a11634562da2 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:58.501085 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac63475-2c02-4ed7-9630-a11634562da2 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:58.501203 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac63475-2c02-4ed7-9630-a11634562da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:58.529493 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-eac63475-2c02-4ed7-9630-a11634562da2 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:58.530793 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-eac63475-2c02-4ed7-9630-a11634562da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:58.608761 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1435/2874] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:58.620160 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678b0a4-071f-463c-8611-f63e59249762 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:58.621825 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a678b0a4-071f-463c-8611-f63e59249762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:58.697812 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678b0a4-071f-463c-8611-f63e59249762 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:58.704603 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7adecd-7ce8-4ae0-92f0-c4450c8bc78c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:58.705293 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7adecd-7ce8-4ae0-92f0-c4450c8bc78c service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:58.705575 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7adecd-7ce8-4ae0-92f0-c4450c8bc78c service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:58.705922 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7adecd-7ce8-4ae0-92f0-c4450c8bc78c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:58.718435 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7adecd-7ce8-4ae0-92f0-c4450c8bc78c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:58.725114 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7adecd-7ce8-4ae0-92f0-c4450c8bc78c service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:58.729751 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:58.737574 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1440/2875] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:08:59.059043 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac63475-2c02-4ed7-9630-a11634562da2 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:59.060526 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-eac63475-2c02-4ed7-9630-a11634562da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:59.080331 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac63475-2c02-4ed7-9630-a11634562da2 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:59.089821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335bc62-22ad-4161-ad47-700da1cbda49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:59.089821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335bc62-22ad-4161-ad47-700da1cbda49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.089821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335bc62-22ad-4161-ad47-700da1cbda49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.089821 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335bc62-22ad-4161-ad47-700da1cbda49 admin admin] PATH_INFO: `/v3/projects/00fb6639cee94a29acf1e3aaff58b8e4` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.091096 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335bc62-22ad-4161-ad47-700da1cbda49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00fb6639cee94a29acf1e3aaff58b8e4)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:59.093430 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335bc62-22ad-4161-ad47-700da1cbda49 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:59.152599 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5335bc62-22ad-4161-ad47-700da1cbda49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00fb6639cee94a29acf1e3aaff58b8e4', 'request_id': 'req-5335bc62-22ad-4161-ad47-700da1cbda49'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:59.160980 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5335bc62-22ad-4161-ad47-700da1cbda49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00fb6639cee94a29acf1e3aaff58b8e4', 'request_id': 'req-5335bc62-22ad-4161-ad47-700da1cbda49'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:59.169582 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5335bc62-22ad-4161-ad47-700da1cbda49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00fb6639cee94a29acf1e3aaff58b8e4', 'request_id': 'req-5335bc62-22ad-4161-ad47-700da1cbda49'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:59.182154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5335bc62-22ad-4161-ad47-700da1cbda49 admin admin] The token cache is being invalidate because project 00fb6639cee94a29acf1e3aaff58b8e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:08:59.182466 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5335bc62-22ad-4161-ad47-700da1cbda49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:59.183104 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-5335bc62-22ad-4161-ad47-700da1cbda49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:59.184894 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1436/2876] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:08:59 2022] DELETE /identity/v3/projects/00fb6639cee94a29acf1e3aaff58b8e4 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:59.189578 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a3086-94d1-4333-b825-73da40a6a8bb None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.190227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a3086-94d1-4333-b825-73da40a6a8bb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.190227 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a3086-94d1-4333-b825-73da40a6a8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.211529 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-cb9a3086-94d1-4333-b825-73da40a6a8bb None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:59.212766 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cb9a3086-94d1-4333-b825-73da40a6a8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:59.275061 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1441/2877] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:08:59.344213 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549aaa28-795b-4fd4-a170-8af9d7dc92e7 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.344505 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549aaa28-795b-4fd4-a170-8af9d7dc92e7 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.344792 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549aaa28-795b-4fd4-a170-8af9d7dc92e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.366554 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-549aaa28-795b-4fd4-a170-8af9d7dc92e7 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:59.367866 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-549aaa28-795b-4fd4-a170-8af9d7dc92e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:59.377435 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1437/2878] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:59.381816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a3086-94d1-4333-b825-73da40a6a8bb None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:59.392403 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a3086-94d1-4333-b825-73da40a6a8bb None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:59.405902 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e33b9-c083-4816-a7a1-7b4bf8cb2805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:59.407212 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e33b9-c083-4816-a7a1-7b4bf8cb2805 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.407785 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e33b9-c083-4816-a7a1-7b4bf8cb2805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.408464 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e33b9-c083-4816-a7a1-7b4bf8cb2805 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.409491 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e33b9-c083-4816-a7a1-7b4bf8cb2805 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:59.410447 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e33b9-c083-4816-a7a1-7b4bf8cb2805 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:59.412490 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e33b9-c083-4816-a7a1-7b4bf8cb2805 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:59.425461 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1442/2879] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 24 09:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:59.430681 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549aaa28-795b-4fd4-a170-8af9d7dc92e7 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:59.459871 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549aaa28-795b-4fd4-a170-8af9d7dc92e7 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:59.465574 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c93c19-601d-4bc2-bb83-4bb88ebbc545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:59.466203 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c93c19-601d-4bc2-bb83-4bb88ebbc545 admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.466478 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c93c19-601d-4bc2-bb83-4bb88ebbc545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.466751 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c93c19-601d-4bc2-bb83-4bb88ebbc545 admin admin] PATH_INFO: `/v3/projects` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.467425 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c93c19-601d-4bc2-bb83-4bb88ebbc545 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:59.468409 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e33b9-c083-4816-a7a1-7b4bf8cb2805 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:59.469860 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c93c19-601d-4bc2-bb83-4bb88ebbc545 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:59.470100 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fa4e33b9-c083-4816-a7a1-7b4bf8cb2805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:59.485191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e33b9-c083-4816-a7a1-7b4bf8cb2805 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:59.491412 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7a03c-005d-4fd3-9307-97c714a1c0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:59.492290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7a03c-005d-4fd3-9307-97c714a1c0f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.492572 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7a03c-005d-4fd3-9307-97c714a1c0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.493677 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7a03c-005d-4fd3-9307-97c714a1c0f4 admin admin] PATH_INFO: `/v3/users/64d2b4e534884384b0997e2fd976581f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.495132 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7a03c-005d-4fd3-9307-97c714a1c0f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64d2b4e534884384b0997e2fd976581f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:59.498041 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7a03c-005d-4fd3-9307-97c714a1c0f4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:59.524189 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1438/2880] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 24 09:08:59 2022] POST /identity/v3/projects => generated 399 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:59.525270 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c93c19-601d-4bc2-bb83-4bb88ebbc545 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:59.565290 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-36c7a03c-005d-4fd3-9307-97c714a1c0f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64d2b4e534884384b0997e2fd976581f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:59.565973 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-36c7a03c-005d-4fd3-9307-97c714a1c0f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64d2b4e534884384b0997e2fd976581f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:59.577943 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-36c7a03c-005d-4fd3-9307-97c714a1c0f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64d2b4e534884384b0997e2fd976581f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:59.591285 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-36c7a03c-005d-4fd3-9307-97c714a1c0f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64d2b4e534884384b0997e2fd976581f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:59.592803 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:59.615955 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c93c19-601d-4bc2-bb83-4bb88ebbc545 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:59.623186 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402fdbc-a95d-4453-9733-2ff0eecddfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:59.624019 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402fdbc-a95d-4453-9733-2ff0eecddfdd admin admin] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.624427 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402fdbc-a95d-4453-9733-2ff0eecddfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.625046 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402fdbc-a95d-4453-9733-2ff0eecddfdd admin admin] PATH_INFO: `/v3/users` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.625272 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-36c7a03c-005d-4fd3-9307-97c714a1c0f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64d2b4e534884384b0997e2fd976581f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:08:59.625928 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402fdbc-a95d-4453-9733-2ff0eecddfdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:59.628270 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402fdbc-a95d-4453-9733-2ff0eecddfdd admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:59.635365 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1443/2881] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:08:59 2022] DELETE /identity/v3/users/64d2b4e534884384b0997e2fd976581f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:59.642549 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e942e-de4e-4967-a60d-3b504f523cd5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.642845 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e942e-de4e-4967-a60d-3b504f523cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.643149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e942e-de4e-4967-a60d-3b504f523cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.658341 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1439/2882] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:59 2022] POST /identity/v3/users => generated 300 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:59.662353 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402fdbc-a95d-4453-9733-2ff0eecddfdd admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:59.664476 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2402fdbc-a95d-4453-9733-2ff0eecddfdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:59.678154 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402fdbc-a95d-4453-9733-2ff0eecddfdd admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:59.680043 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c35e942e-de4e-4967-a60d-3b504f523cd5 None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:59.683655 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c35e942e-de4e-4967-a60d-3b504f523cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:59.764224 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c0c04-0f83-4e10-8f11-013df870960e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:59.765545 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c0c04-0f83-4e10-8f11-013df870960e admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.766410 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c0c04-0f83-4e10-8f11-013df870960e admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.767934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c0c04-0f83-4e10-8f11-013df870960e admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.769002 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c0c04-0f83-4e10-8f11-013df870960e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:59.773139 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c0c04-0f83-4e10-8f11-013df870960e admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:59.784489 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:59.785044 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1440/2883] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:59.790840 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c0c04-0f83-4e10-8f11-013df870960e admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:59.804185 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1444/2884] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:08:59.805131 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c0c04-0f83-4e10-8f11-013df870960e admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:59.810690 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252dee35-8163-49e2-ac39-5104b068db22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:59.811377 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252dee35-8163-49e2-ac39-5104b068db22 admin admin] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.811724 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252dee35-8163-49e2-ac39-5104b068db22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.812063 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252dee35-8163-49e2-ac39-5104b068db22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.812811 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252dee35-8163-49e2-ac39-5104b068db22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:59.814954 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252dee35-8163-49e2-ac39-5104b068db22 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:59.815478 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e942e-de4e-4967-a60d-3b504f523cd5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:59.818031 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c35e942e-de4e-4967-a60d-3b504f523cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:59.829023 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1441/2885] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 24 09:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:08:59.832692 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252dee35-8163-49e2-ac39-5104b068db22 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:59.844493 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252dee35-8163-49e2-ac39-5104b068db22 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:59.854906 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28626121-9c1c-4b7f-856b-5a5c489aa980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:59.855619 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28626121-9c1c-4b7f-856b-5a5c489aa980 admin admin] REQUEST_METHOD: `PUT` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.855901 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28626121-9c1c-4b7f-856b-5a5c489aa980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.856446 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28626121-9c1c-4b7f-856b-5a5c489aa980 admin admin] PATH_INFO: `/v3/projects/bce90e8f4fec46d6960b9181e05ea2d8/users/3bacd9a18323462193063724cd5b0e69/roles/92533f29c1d54fe09591e6f2a906ac04` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.874660 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28626121-9c1c-4b7f-856b-5a5c489aa980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bce90e8f4fec46d6960b9181e05ea2d8, user_id=3bacd9a18323462193063724cd5b0e69, role_id=92533f29c1d54fe09591e6f2a906ac04)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:59.877606 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28626121-9c1c-4b7f-856b-5a5c489aa980 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:59.902725 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1442/2886] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 24 09:08:59 2022] PUT /identity/v3/projects/bce90e8f4fec46d6960b9181e05ea2d8/users/3bacd9a18323462193063724cd5b0e69/roles/92533f29c1d54fe09591e6f2a906ac04 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:59.913410 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5f6a-d2b5-4408-8add-36e4fe1f3551 None None] REQUEST_METHOD: `POST` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.914513 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5f6a-d2b5-4408-8add-36e4fe1f3551 None None] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.914970 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5f6a-d2b5-4408-8add-36e4fe1f3551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.927977 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e942e-de4e-4967-a60d-3b504f523cd5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:59.951343 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.auth.core [None req-d38b5f6a-d2b5-4408-8add-36e4fe1f3551 None None] MFA Rules not processed for user `3bacd9a18323462193063724cd5b0e69`. Rule list: `[]` (Enabled: `True`). {{(pid=81476) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:59.952706 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d38b5f6a-d2b5-4408-8add-36e4fe1f3551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:59.980107 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7629413-bf9b-4196-b9ba-9b156275e88c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:59.980738 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7629413-bf9b-4196-b9ba-9b156275e88c service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.981017 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7629413-bf9b-4196-b9ba-9b156275e88c service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.981289 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7629413-bf9b-4196-b9ba-9b156275e88c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:00.024773 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1443/2887] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:08:59 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:00.036528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b5f6a-d2b5-4408-8add-36e4fe1f3551 None None] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:00.038261 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-d38b5f6a-d2b5-4408-8add-36e4fe1f3551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:00.051168 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7629413-bf9b-4196-b9ba-9b156275e88c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:00.052822 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7629413-bf9b-4196-b9ba-9b156275e88c service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:00.062941 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1445/2888] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:08:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:00.073206 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:00.102799 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b5f6a-d2b5-4408-8add-36e4fe1f3551 None None] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:00.106488 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9949808-3559-430e-96a0-2a75b7b2b5a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5de1586efc8c4dec898b0fb54b56acf0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:00.107104 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9949808-3559-430e-96a0-2a75b7b2b5a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:00.107382 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9949808-3559-430e-96a0-2a75b7b2b5a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:00.107655 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9949808-3559-430e-96a0-2a75b7b2b5a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:00.116445 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9949808-3559-430e-96a0-2a75b7b2b5a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:00.118167 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9949808-3559-430e-96a0-2a75b7b2b5a9 service cinder] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:00.128070 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1444/2889] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:00 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:00.352776 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7629413-bf9b-4196-b9ba-9b156275e88c service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:00.354718 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c7629413-bf9b-4196-b9ba-9b156275e88c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:00.370494 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7629413-bf9b-4196-b9ba-9b156275e88c service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:00.375717 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bbbfb-490c-4d62-8191-4f2ce61ad18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:00.376422 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bbbfb-490c-4d62-8191-4f2ce61ad18d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:00.376700 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bbbfb-490c-4d62-8191-4f2ce61ad18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:00.376969 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bbbfb-490c-4d62-8191-4f2ce61ad18d admin admin] PATH_INFO: `/v3/projects/7c4762824948495781926df2399c75c1` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:00.378622 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1bbbfb-490c-4d62-8191-4f2ce61ad18d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c4762824948495781926df2399c75c1)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:00.380191 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1bbbfb-490c-4d62-8191-4f2ce61ad18d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:00.428267 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3b1bbbfb-490c-4d62-8191-4f2ce61ad18d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c4762824948495781926df2399c75c1', 'request_id': 'req-3b1bbbfb-490c-4d62-8191-4f2ce61ad18d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:00.435954 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3b1bbbfb-490c-4d62-8191-4f2ce61ad18d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c4762824948495781926df2399c75c1', 'request_id': 'req-3b1bbbfb-490c-4d62-8191-4f2ce61ad18d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:00.437938 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:00.445483 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3b1bbbfb-490c-4d62-8191-4f2ce61ad18d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c4762824948495781926df2399c75c1', 'request_id': 'req-3b1bbbfb-490c-4d62-8191-4f2ce61ad18d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:00.457511 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3b1bbbfb-490c-4d62-8191-4f2ce61ad18d admin admin] The token cache is being invalidate because project 7c4762824948495781926df2399c75c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:09:00.457872 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3b1bbbfb-490c-4d62-8191-4f2ce61ad18d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:00.458593 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3b1bbbfb-490c-4d62-8191-4f2ce61ad18d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:00.460170 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1446/2890] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:09:00 2022] DELETE /identity/v3/projects/7c4762824948495781926df2399c75c1 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:10.123835 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=14 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:10.454351 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:17.583872 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9949808-3559-430e-96a0-2a75b7b2b5a9 service cinder] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:17.594367 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9949808-3559-430e-96a0-2a75b7b2b5a9 service cinder] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:17.649868 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c0fae-8a84-454b-9a01-5d95a423cfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:17.650493 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c0fae-8a84-454b-9a01-5d95a423cfa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:17.650772 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c0fae-8a84-454b-9a01-5d95a423cfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:17.651043 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c0fae-8a84-454b-9a01-5d95a423cfa2 admin admin] PATH_INFO: `/v3/users/3bacd9a18323462193063724cd5b0e69` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:17.652182 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c0fae-8a84-454b-9a01-5d95a423cfa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bacd9a18323462193063724cd5b0e69)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:17.654101 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c0fae-8a84-454b-9a01-5d95a423cfa2 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:17.700752 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-0a0c0fae-8a84-454b-9a01-5d95a423cfa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bacd9a18323462193063724cd5b0e69'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:17.707961 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-0a0c0fae-8a84-454b-9a01-5d95a423cfa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bacd9a18323462193063724cd5b0e69'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:17.719929 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-0a0c0fae-8a84-454b-9a01-5d95a423cfa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bacd9a18323462193063724cd5b0e69'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:17.727406 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-0a0c0fae-8a84-454b-9a01-5d95a423cfa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bacd9a18323462193063724cd5b0e69'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:17.728110 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-0a0c0fae-8a84-454b-9a01-5d95a423cfa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bacd9a18323462193063724cd5b0e69'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:17.733857 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1445/2891] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:09:17 2022] DELETE /identity/v3/users/3bacd9a18323462193063724cd5b0e69 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:17.738336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa4192-a093-41db-9f6f-0d5a9bef665b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:17.738867 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa4192-a093-41db-9f6f-0d5a9bef665b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:17.738867 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa4192-a093-41db-9f6f-0d5a9bef665b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:17.758487 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-00fa4192-a093-41db-9f6f-0d5a9bef665b None None] MFA Rules not processed for user `d66e21fb9473433681e9a7a5e248373c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:17.759685 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-00fa4192-a093-41db-9f6f-0d5a9bef665b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:17.820238 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1447/2892] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 24 09:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:09:17.830060 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c0fae-8a84-454b-9a01-5d95a423cfa2 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:17.831498 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0a0c0fae-8a84-454b-9a01-5d95a423cfa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:17.889185 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c0fae-8a84-454b-9a01-5d95a423cfa2 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:17.897423 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366232d-a346-485e-81b8-d6438154b422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:17.898026 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366232d-a346-485e-81b8-d6438154b422 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:17.898304 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366232d-a346-485e-81b8-d6438154b422 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:17.898573 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366232d-a346-485e-81b8-d6438154b422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:17.907573 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366232d-a346-485e-81b8-d6438154b422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:17.909104 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366232d-a346-485e-81b8-d6438154b422 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:17.919127 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1446/2893] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:09:18.182421 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa4192-a093-41db-9f6f-0d5a9bef665b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:18.183827 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-00fa4192-a093-41db-9f6f-0d5a9bef665b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:18.197013 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa4192-a093-41db-9f6f-0d5a9bef665b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:18.201723 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1fa1b-fa0b-4daa-8e53-0e97826c2bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:18.202383 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1fa1b-fa0b-4daa-8e53-0e97826c2bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:18.202661 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1fa1b-fa0b-4daa-8e53-0e97826c2bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:18.202931 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1fa1b-fa0b-4daa-8e53-0e97826c2bf6 admin admin] PATH_INFO: `/v3/projects/bce90e8f4fec46d6960b9181e05ea2d8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:18.204081 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e1fa1b-fa0b-4daa-8e53-0e97826c2bf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bce90e8f4fec46d6960b9181e05ea2d8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:18.206074 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e1fa1b-fa0b-4daa-8e53-0e97826c2bf6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:18.259336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-51e1fa1b-fa0b-4daa-8e53-0e97826c2bf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bce90e8f4fec46d6960b9181e05ea2d8', 'request_id': 'req-51e1fa1b-fa0b-4daa-8e53-0e97826c2bf6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:18.268074 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-51e1fa1b-fa0b-4daa-8e53-0e97826c2bf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bce90e8f4fec46d6960b9181e05ea2d8', 'request_id': 'req-51e1fa1b-fa0b-4daa-8e53-0e97826c2bf6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:18.280147 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-51e1fa1b-fa0b-4daa-8e53-0e97826c2bf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bce90e8f4fec46d6960b9181e05ea2d8', 'request_id': 'req-51e1fa1b-fa0b-4daa-8e53-0e97826c2bf6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:18.293930 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-51e1fa1b-fa0b-4daa-8e53-0e97826c2bf6 admin admin] The token cache is being invalidate because project bce90e8f4fec46d6960b9181e05ea2d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:09:18.294381 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-51e1fa1b-fa0b-4daa-8e53-0e97826c2bf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:18.295164 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-51e1fa1b-fa0b-4daa-8e53-0e97826c2bf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:18.297175 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1448/2894] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:09:18 2022] DELETE /identity/v3/projects/bce90e8f4fec46d6960b9181e05ea2d8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:18.626918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366232d-a346-485e-81b8-d6438154b422 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:18.628379 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-1366232d-a346-485e-81b8-d6438154b422 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:18.655200 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:18.719202 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366232d-a346-485e-81b8-d6438154b422 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:18.785188 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f216fb-4fb4-4bb6-88f9-86b7157d7d9e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33a0844a1c8f4ea4932e12d53e5ee9ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:18.785804 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f216fb-4fb4-4bb6-88f9-86b7157d7d9e service placement] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:18.786604 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f216fb-4fb4-4bb6-88f9-86b7157d7d9e service placement] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:18.786967 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f216fb-4fb4-4bb6-88f9-86b7157d7d9e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:18.788737 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b0f216fb-4fb4-4bb6-88f9-86b7157d7d9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:18.800949 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f216fb-4fb4-4bb6-88f9-86b7157d7d9e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:18.803496 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f216fb-4fb4-4bb6-88f9-86b7157d7d9e service placement] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:18.812896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1447/2895] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:22.447041 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1fa1b-fa0b-4daa-8e53-0e97826c2bf6 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:22.449359 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-51e1fa1b-fa0b-4daa-8e53-0e97826c2bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:22.529623 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1fa1b-fa0b-4daa-8e53-0e97826c2bf6 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:22.536633 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed8539-96f9-4901-adda-ef93006663e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:22.537230 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed8539-96f9-4901-adda-ef93006663e4 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:22.537513 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed8539-96f9-4901-adda-ef93006663e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:22.537775 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed8539-96f9-4901-adda-ef93006663e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:22.539559 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-97ed8539-96f9-4901-adda-ef93006663e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:22.552753 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed8539-96f9-4901-adda-ef93006663e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:22.554335 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed8539-96f9-4901-adda-ef93006663e4 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:22.564892 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1449/2896] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:28.789472 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f216fb-4fb4-4bb6-88f9-86b7157d7d9e service placement] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:28.790974 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b0f216fb-4fb4-4bb6-88f9-86b7157d7d9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:28.853808 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f216fb-4fb4-4bb6-88f9-86b7157d7d9e service placement] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:28.857365 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2bfad-3e1c-4311-9d73-6c8a2ee5692b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:28.857968 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2bfad-3e1c-4311-9d73-6c8a2ee5692b service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:28.858248 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2bfad-3e1c-4311-9d73-6c8a2ee5692b service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:28.858528 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2bfad-3e1c-4311-9d73-6c8a2ee5692b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:28.860280 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-4ab2bfad-3e1c-4311-9d73-6c8a2ee5692b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:28.922769 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2bfad-3e1c-4311-9d73-6c8a2ee5692b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:28.924241 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2bfad-3e1c-4311-9d73-6c8a2ee5692b service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:28.934577 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1448/2897] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:32.560446 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:35.372018 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed8539-96f9-4901-adda-ef93006663e4 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:35.373625 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-97ed8539-96f9-4901-adda-ef93006663e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:35.394155 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed8539-96f9-4901-adda-ef93006663e4 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:35.398707 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ff433-e14d-4ed5-a92b-def0eee8195e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:35.399336 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ff433-e14d-4ed5-a92b-def0eee8195e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:35.399617 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ff433-e14d-4ed5-a92b-def0eee8195e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:35.399891 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ff433-e14d-4ed5-a92b-def0eee8195e admin admin] PATH_INFO: `/v3/users/46b3aa99a5e84ef2a8df193628feb0d6` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:35.414024 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ff433-e14d-4ed5-a92b-def0eee8195e admin admin] RBAC: Authorizing `identity:delete_user(user_id=46b3aa99a5e84ef2a8df193628feb0d6)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:35.415944 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ff433-e14d-4ed5-a92b-def0eee8195e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:35.460800 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-155ff433-e14d-4ed5-a92b-def0eee8195e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46b3aa99a5e84ef2a8df193628feb0d6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:35.461431 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-155ff433-e14d-4ed5-a92b-def0eee8195e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46b3aa99a5e84ef2a8df193628feb0d6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:35.468360 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-155ff433-e14d-4ed5-a92b-def0eee8195e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46b3aa99a5e84ef2a8df193628feb0d6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:35.475230 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-155ff433-e14d-4ed5-a92b-def0eee8195e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46b3aa99a5e84ef2a8df193628feb0d6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:35.486707 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-155ff433-e14d-4ed5-a92b-def0eee8195e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46b3aa99a5e84ef2a8df193628feb0d6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:35.493301 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1450/2898] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:09:35 2022] DELETE /identity/v3/users/46b3aa99a5e84ef2a8df193628feb0d6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:35.671893 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2bfad-3e1c-4311-9d73-6c8a2ee5692b service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:35.679683 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=100 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:35.683856 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2bfad-3e1c-4311-9d73-6c8a2ee5692b service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:35.743220 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856bd66-c9ce-4bce-9ef1-886df57527af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:35.743854 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856bd66-c9ce-4bce-9ef1-886df57527af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:35.744159 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856bd66-c9ce-4bce-9ef1-886df57527af admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:35.744434 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856bd66-c9ce-4bce-9ef1-886df57527af admin admin] PATH_INFO: `/v3/projects/080318594f144fa3852577bc02ee5357` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:35.756190 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856bd66-c9ce-4bce-9ef1-886df57527af admin admin] RBAC: Authorizing `identity:delete_project(project_id=080318594f144fa3852577bc02ee5357)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:35.758878 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856bd66-c9ce-4bce-9ef1-886df57527af admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:35.812985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2856bd66-c9ce-4bce-9ef1-886df57527af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '080318594f144fa3852577bc02ee5357', 'request_id': 'req-2856bd66-c9ce-4bce-9ef1-886df57527af'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:35.820688 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2856bd66-c9ce-4bce-9ef1-886df57527af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '080318594f144fa3852577bc02ee5357', 'request_id': 'req-2856bd66-c9ce-4bce-9ef1-886df57527af'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:35.827166 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2856bd66-c9ce-4bce-9ef1-886df57527af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '080318594f144fa3852577bc02ee5357', 'request_id': 'req-2856bd66-c9ce-4bce-9ef1-886df57527af'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:35.838772 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2856bd66-c9ce-4bce-9ef1-886df57527af admin admin] The token cache is being invalidate because project 080318594f144fa3852577bc02ee5357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81476) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:09:35.839062 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2856bd66-c9ce-4bce-9ef1-886df57527af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:35.839699 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-2856bd66-c9ce-4bce-9ef1-886df57527af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:35.841216 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1449/2899] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:09:35 2022] DELETE /identity/v3/projects/080318594f144fa3852577bc02ee5357 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:35.845430 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ff433-e14d-4ed5-a92b-def0eee8195e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:35.848051 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-155ff433-e14d-4ed5-a92b-def0eee8195e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:35.861369 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ff433-e14d-4ed5-a92b-def0eee8195e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:35.919012 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa55467-aaf1-489c-af27-af0482ff5675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:35.919650 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa55467-aaf1-489c-af27-af0482ff5675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:35.919926 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa55467-aaf1-489c-af27-af0482ff5675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:35.920233 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa55467-aaf1-489c-af27-af0482ff5675 admin admin] PATH_INFO: `/v3/users/fd47057b9c334e1eb36ff49bd3273171` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:35.935246 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa55467-aaf1-489c-af27-af0482ff5675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd47057b9c334e1eb36ff49bd3273171)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:35.937719 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa55467-aaf1-489c-af27-af0482ff5675 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:35.991876 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0aa55467-aaf1-489c-af27-af0482ff5675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd47057b9c334e1eb36ff49bd3273171'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:35.992559 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0aa55467-aaf1-489c-af27-af0482ff5675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd47057b9c334e1eb36ff49bd3273171'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:35.996380 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:36.000663 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0aa55467-aaf1-489c-af27-af0482ff5675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd47057b9c334e1eb36ff49bd3273171'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:36.010085 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0aa55467-aaf1-489c-af27-af0482ff5675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd47057b9c334e1eb36ff49bd3273171'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:36.023117 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0aa55467-aaf1-489c-af27-af0482ff5675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd47057b9c334e1eb36ff49bd3273171'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:36.029407 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1451/2900] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:09:35 2022] DELETE /identity/v3/users/fd47057b9c334e1eb36ff49bd3273171 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:36.122733 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856bd66-c9ce-4bce-9ef1-886df57527af admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:36.124249 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-2856bd66-c9ce-4bce-9ef1-886df57527af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:36.181378 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856bd66-c9ce-4bce-9ef1-886df57527af admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:36.210206 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa55467-aaf1-489c-af27-af0482ff5675 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:36.212559 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0aa55467-aaf1-489c-af27-af0482ff5675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:36.230896 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:36.247297 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14777e10-da46-4eab-badd-bc65bb70fdad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:36.248390 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14777e10-da46-4eab-badd-bc65bb70fdad service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:36.248390 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14777e10-da46-4eab-badd-bc65bb70fdad service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:36.248598 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14777e10-da46-4eab-badd-bc65bb70fdad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:36.250497 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-14777e10-da46-4eab-badd-bc65bb70fdad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:36.293490 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa55467-aaf1-489c-af27-af0482ff5675 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:36.300223 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9ca8a-beaf-4e5d-84d4-611b10e609ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:36.301255 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9ca8a-beaf-4e5d-84d4-611b10e609ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:36.301540 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9ca8a-beaf-4e5d-84d4-611b10e609ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:36.301816 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9ca8a-beaf-4e5d-84d4-611b10e609ec admin admin] PATH_INFO: `/v3/projects/364a8d179c244dcd83327a69ec26c5e0` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:36.313400 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9ca8a-beaf-4e5d-84d4-611b10e609ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=364a8d179c244dcd83327a69ec26c5e0)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:36.315991 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9ca8a-beaf-4e5d-84d4-611b10e609ec admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:36.323826 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14777e10-da46-4eab-badd-bc65bb70fdad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:36.325890 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14777e10-da46-4eab-badd-bc65bb70fdad service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:36.336439 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1450/2901] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:09:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:09:36.368801 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e2e9ca8a-beaf-4e5d-84d4-611b10e609ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '364a8d179c244dcd83327a69ec26c5e0', 'request_id': 'req-e2e9ca8a-beaf-4e5d-84d4-611b10e609ec'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:36.374699 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e2e9ca8a-beaf-4e5d-84d4-611b10e609ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '364a8d179c244dcd83327a69ec26c5e0', 'request_id': 'req-e2e9ca8a-beaf-4e5d-84d4-611b10e609ec'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:36.382715 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e2e9ca8a-beaf-4e5d-84d4-611b10e609ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '364a8d179c244dcd83327a69ec26c5e0', 'request_id': 'req-e2e9ca8a-beaf-4e5d-84d4-611b10e609ec'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:36.396721 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e2e9ca8a-beaf-4e5d-84d4-611b10e609ec admin admin] The token cache is being invalidate because project 364a8d179c244dcd83327a69ec26c5e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:09:36.396721 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e2e9ca8a-beaf-4e5d-84d4-611b10e609ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:36.397553 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e2e9ca8a-beaf-4e5d-84d4-611b10e609ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:09:36.399140 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1452/2902] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:09:36 2022] DELETE /identity/v3/projects/364a8d179c244dcd83327a69ec26c5e0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:46.331553 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=31 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:46.393035 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=67,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:40.439920 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14777e10-da46-4eab-badd-bc65bb70fdad service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:40.441585 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-14777e10-da46-4eab-badd-bc65bb70fdad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:40.454019 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14777e10-da46-4eab-badd-bc65bb70fdad service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:40.507054 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472f434-a09a-42a0-9853-b59c5d7700b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:40.507650 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472f434-a09a-42a0-9853-b59c5d7700b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:40.507932 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472f434-a09a-42a0-9853-b59c5d7700b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:40.508228 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472f434-a09a-42a0-9853-b59c5d7700b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:40.509967 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-0472f434-a09a-42a0-9853-b59c5d7700b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:40.590952 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0472f434-a09a-42a0-9853-b59c5d7700b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:40.592413 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0472f434-a09a-42a0-9853-b59c5d7700b9 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:40.603966 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1451/2903] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:10:41.887818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9ca8a-beaf-4e5d-84d4-611b10e609ec admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:41.889248 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e2e9ca8a-beaf-4e5d-84d4-611b10e609ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:41.901956 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9ca8a-beaf-4e5d-84d4-611b10e609ec admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:41.905502 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433ecac-9fe2-4d35-b3e7-dad72be82d62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:41.906099 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433ecac-9fe2-4d35-b3e7-dad72be82d62 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:41.906377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433ecac-9fe2-4d35-b3e7-dad72be82d62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:41.906652 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433ecac-9fe2-4d35-b3e7-dad72be82d62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:41.908372 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4433ecac-9fe2-4d35-b3e7-dad72be82d62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:41.921872 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433ecac-9fe2-4d35-b3e7-dad72be82d62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:41.923275 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433ecac-9fe2-4d35-b3e7-dad72be82d62 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:41.934131 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1453/2904] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:10:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:10:50.598581 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=54 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:51.929222 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:48.498565 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472f434-a09a-42a0-9853-b59c5d7700b9 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:48.507593 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472f434-a09a-42a0-9853-b59c5d7700b9 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:48.511934 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ccc8b-a218-4bfc-95d7-56613d378d50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ee2e93901914b1d82607a3ae0e2acf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:48.512573 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ccc8b-a218-4bfc-95d7-56613d378d50 service neutron] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:48.512855 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ccc8b-a218-4bfc-95d7-56613d378d50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:48.513125 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ccc8b-a218-4bfc-95d7-56613d378d50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:48.514815 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-406ccc8b-a218-4bfc-95d7-56613d378d50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:48.524339 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ccc8b-a218-4bfc-95d7-56613d378d50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:48.526553 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ccc8b-a218-4bfc-95d7-56613d378d50 service neutron] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:48.534450 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1452/2905] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:11:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:11:56.715132 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433ecac-9fe2-4d35-b3e7-dad72be82d62 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:56.717029 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4433ecac-9fe2-4d35-b3e7-dad72be82d62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:56.745605 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433ecac-9fe2-4d35-b3e7-dad72be82d62 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:56.751523 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e8012-e71f-4b1f-ae62-fe059cf5d951 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:56.752472 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e8012-e71f-4b1f-ae62-fe059cf5d951 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:56.752818 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e8012-e71f-4b1f-ae62-fe059cf5d951 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:56.753187 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e8012-e71f-4b1f-ae62-fe059cf5d951 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:56.755260 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-795e8012-e71f-4b1f-ae62-fe059cf5d951 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:56.769514 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e8012-e71f-4b1f-ae62-fe059cf5d951 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:56.771172 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e8012-e71f-4b1f-ae62-fe059cf5d951 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:56.783950 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1454/2906] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:11:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:11:58.115128 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ccc8b-a218-4bfc-95d7-56613d378d50 service neutron] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:58.129654 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ccc8b-a218-4bfc-95d7-56613d378d50 service neutron] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:58.136923 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e3bf1-768e-4fcd-8875-b83d6476186a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca56dd7216fb4946946b2801e144fec2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:58.137540 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e3bf1-768e-4fcd-8875-b83d6476186a service nova] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:58.137815 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e3bf1-768e-4fcd-8875-b83d6476186a service nova] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:58.138078 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e3bf1-768e-4fcd-8875-b83d6476186a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:58.140625 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3f2e3bf1-768e-4fcd-8875-b83d6476186a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:58.154850 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e3bf1-768e-4fcd-8875-b83d6476186a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:58.156571 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e3bf1-768e-4fcd-8875-b83d6476186a service nova] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:58.169660 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1453/2907] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:11:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:12:05.333538 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f11648-ae92-41a2-9980-67ee6f8b850f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:05.334036 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f11648-ae92-41a2-9980-67ee6f8b850f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:05.334149 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f11648-ae92-41a2-9980-67ee6f8b850f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:05.353767 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e6f11648-ae92-41a2-9980-67ee6f8b850f None None] MFA Rules not processed for user `b11abed8d4424a1e9957c3dc9aefadd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:12:05.354951 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e6f11648-ae92-41a2-9980-67ee6f8b850f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:05.408585 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1455/2908] 10.222.0.67 () {58 vars in 967 bytes} [Wed Aug 24 09:12:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:12:05.449818 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e3bf1-768e-4fcd-8875-b83d6476186a service nova] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:05.451490 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-3f2e3bf1-768e-4fcd-8875-b83d6476186a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:05.518625 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e3bf1-768e-4fcd-8875-b83d6476186a service nova] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:05.523514 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ef2d8-1661-4be0-9a25-c9ea1eb98fb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46b1417d4ac24c8982b2052f4e99042a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:05.524136 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ef2d8-1661-4be0-9a25-c9ea1eb98fb8 service swift] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:05.524419 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ef2d8-1661-4be0-9a25-c9ea1eb98fb8 service swift] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:05.524717 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ef2d8-1661-4be0-9a25-c9ea1eb98fb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:05.532550 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ef2d8-1661-4be0-9a25-c9ea1eb98fb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:05.534053 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ef2d8-1661-4be0-9a25-c9ea1eb98fb8 service swift] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:05.542342 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1454/2909] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 24 09:12:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:12:11.898618 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f11648-ae92-41a2-9980-67ee6f8b850f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:11.900413 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e6f11648-ae92-41a2-9980-67ee6f8b850f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:11.965197 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ef2d8-1661-4be0-9a25-c9ea1eb98fb8 service swift] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:11.966757 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-7c2ef2d8-1661-4be0-9a25-c9ea1eb98fb8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:11.972473 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=45 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:12:11.980831 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f11648-ae92-41a2-9980-67ee6f8b850f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:11.985910 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52cbc0-0c8f-4c99-9e0a-688917e4c3a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33a0844a1c8f4ea4932e12d53e5ee9ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:11.986576 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52cbc0-0c8f-4c99-9e0a-688917e4c3a1 service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:11.986806 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52cbc0-0c8f-4c99-9e0a-688917e4c3a1 service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:11.987079 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52cbc0-0c8f-4c99-9e0a-688917e4c3a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:11.988835 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5e52cbc0-0c8f-4c99-9e0a-688917e4c3a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:12.001355 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52cbc0-0c8f-4c99-9e0a-688917e4c3a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:12.002935 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52cbc0-0c8f-4c99-9e0a-688917e4c3a1 service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:12.013220 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1456/2910] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:12:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:12:12.042055 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ef2d8-1661-4be0-9a25-c9ea1eb98fb8 service swift] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:12.047344 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1d2a6-643b-40c2-9765-dee2e1eda485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:12.047976 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1d2a6-643b-40c2-9765-dee2e1eda485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:12.048293 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1d2a6-643b-40c2-9765-dee2e1eda485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:12.048566 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1d2a6-643b-40c2-9765-dee2e1eda485 admin admin] PATH_INFO: `/v3/users/517a70383bcd42eba23483981dd40d89` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:12.049941 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f1d2a6-643b-40c2-9765-dee2e1eda485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=517a70383bcd42eba23483981dd40d89)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:12.051852 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f1d2a6-643b-40c2-9765-dee2e1eda485 admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:12.104322 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-60f1d2a6-643b-40c2-9765-dee2e1eda485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '517a70383bcd42eba23483981dd40d89'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.112135 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-60f1d2a6-643b-40c2-9765-dee2e1eda485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '517a70383bcd42eba23483981dd40d89'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.126809 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-60f1d2a6-643b-40c2-9765-dee2e1eda485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '517a70383bcd42eba23483981dd40d89'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.133928 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-60f1d2a6-643b-40c2-9765-dee2e1eda485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '517a70383bcd42eba23483981dd40d89'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.134623 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-60f1d2a6-643b-40c2-9765-dee2e1eda485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '517a70383bcd42eba23483981dd40d89'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.140463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1455/2911] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:12:11 2022] DELETE /identity/v3/users/517a70383bcd42eba23483981dd40d89 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:12:12.314288 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52cbc0-0c8f-4c99-9e0a-688917e4c3a1 service placement] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:12.328234 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52cbc0-0c8f-4c99-9e0a-688917e4c3a1 service placement] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:12.392021 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09d8d3-a792-4a5d-9d99-6d9447029650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:12.392693 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09d8d3-a792-4a5d-9d99-6d9447029650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:12.392970 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09d8d3-a792-4a5d-9d99-6d9447029650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:12.393240 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09d8d3-a792-4a5d-9d99-6d9447029650 admin admin] PATH_INFO: `/v3/projects/759df091a0664e68b9f38eec1f9c2e49` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:12.394387 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09d8d3-a792-4a5d-9d99-6d9447029650 admin admin] RBAC: Authorizing `identity:delete_project(project_id=759df091a0664e68b9f38eec1f9c2e49)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:12.396478 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09d8d3-a792-4a5d-9d99-6d9447029650 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:12.423749 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:12:12.450069 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4d09d8d3-a792-4a5d-9d99-6d9447029650 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '759df091a0664e68b9f38eec1f9c2e49', 'request_id': 'req-4d09d8d3-a792-4a5d-9d99-6d9447029650'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.456196 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4d09d8d3-a792-4a5d-9d99-6d9447029650 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '759df091a0664e68b9f38eec1f9c2e49', 'request_id': 'req-4d09d8d3-a792-4a5d-9d99-6d9447029650'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.465362 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4d09d8d3-a792-4a5d-9d99-6d9447029650 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '759df091a0664e68b9f38eec1f9c2e49', 'request_id': 'req-4d09d8d3-a792-4a5d-9d99-6d9447029650'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.477000 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4d09d8d3-a792-4a5d-9d99-6d9447029650 admin admin] The token cache is being invalidate because project 759df091a0664e68b9f38eec1f9c2e49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:12:12.477898 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4d09d8d3-a792-4a5d-9d99-6d9447029650 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.478743 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4d09d8d3-a792-4a5d-9d99-6d9447029650 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.480287 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1457/2912] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:12:12 2022] DELETE /identity/v3/projects/759df091a0664e68b9f38eec1f9c2e49 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:12:12.484419 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1d2a6-643b-40c2-9765-dee2e1eda485 admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:12.485853 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-60f1d2a6-643b-40c2-9765-dee2e1eda485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:12.499066 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1d2a6-643b-40c2-9765-dee2e1eda485 admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:12.554375 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3f08f-5707-41de-b2c9-b28f515d166f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:12.554999 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3f08f-5707-41de-b2c9-b28f515d166f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:12.555315 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3f08f-5707-41de-b2c9-b28f515d166f admin admin] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:12.555668 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3f08f-5707-41de-b2c9-b28f515d166f admin admin] PATH_INFO: `/v3/users/ba92b7c8f75e4977a4164308dbbcbbfc` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:12.556902 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3f08f-5707-41de-b2c9-b28f515d166f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba92b7c8f75e4977a4164308dbbcbbfc)` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:12.558855 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3f08f-5707-41de-b2c9-b28f515d166f admin admin] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:12.581463 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:12:12.607449 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b3d3f08f-5707-41de-b2c9-b28f515d166f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba92b7c8f75e4977a4164308dbbcbbfc'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.613992 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b3d3f08f-5707-41de-b2c9-b28f515d166f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba92b7c8f75e4977a4164308dbbcbbfc'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.626780 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b3d3f08f-5707-41de-b2c9-b28f515d166f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba92b7c8f75e4977a4164308dbbcbbfc'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.633543 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b3d3f08f-5707-41de-b2c9-b28f515d166f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba92b7c8f75e4977a4164308dbbcbbfc'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.634177 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.notifications [None req-b3d3f08f-5707-41de-b2c9-b28f515d166f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba92b7c8f75e4977a4164308dbbcbbfc'} {{(pid=81476) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.639943 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1456/2913] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 24 09:12:12 2022] DELETE /identity/v3/users/ba92b7c8f75e4977a4164308dbbcbbfc => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:12:12.827989 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09d8d3-a792-4a5d-9d99-6d9447029650 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:12.830344 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4d09d8d3-a792-4a5d-9d99-6d9447029650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:12.848393 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09d8d3-a792-4a5d-9d99-6d9447029650 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:12.913006 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759de338-7203-4410-998c-cc26220c3602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd66e21fb9473433681e9a7a5e248373c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '456ae4386c984b51b020a86192cad430', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:12.913735 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759de338-7203-4410-998c-cc26220c3602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:12.913909 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759de338-7203-4410-998c-cc26220c3602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:12.914208 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759de338-7203-4410-998c-cc26220c3602 admin admin] PATH_INFO: `/v3/projects/0ce48c20a82e4188844001db37794dde` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:12.916178 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759de338-7203-4410-998c-cc26220c3602 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ce48c20a82e4188844001db37794dde)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:12.918071 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759de338-7203-4410-998c-cc26220c3602 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:12.963807 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-759de338-7203-4410-998c-cc26220c3602 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ce48c20a82e4188844001db37794dde', 'request_id': 'req-759de338-7203-4410-998c-cc26220c3602'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.969252 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-759de338-7203-4410-998c-cc26220c3602 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ce48c20a82e4188844001db37794dde', 'request_id': 'req-759de338-7203-4410-998c-cc26220c3602'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.976281 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-759de338-7203-4410-998c-cc26220c3602 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ce48c20a82e4188844001db37794dde', 'request_id': 'req-759de338-7203-4410-998c-cc26220c3602'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.987097 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-759de338-7203-4410-998c-cc26220c3602 admin admin] The token cache is being invalidate because project 0ce48c20a82e4188844001db37794dde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:12:12.987377 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-759de338-7203-4410-998c-cc26220c3602 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.988008 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-759de338-7203-4410-998c-cc26220c3602 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:12:12.989513 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1458/2914] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 24 09:12:12 2022] DELETE /identity/v3/projects/0ce48c20a82e4188844001db37794dde => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:12:22.636544 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=20,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:12:22.984276 dv-cinder-835384-23-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=66,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:14:47.726555 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3f08f-5707-41de-b2c9-b28f515d166f admin admin] Authenticating user token {{(pid=81476) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:14:47.727985 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-b3d3f08f-5707-41de-b2c9-b28f515d166f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:14:47.868822 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3f08f-5707-41de-b2c9-b28f515d166f admin admin] Validating token access rules against request {{(pid=81476) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:14:47.927305 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608e10b-7bdf-4158-9164-85e9102aff86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33a0844a1c8f4ea4932e12d53e5ee9ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fce6ee19f58b4a468c9a5d169e29a018', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81476) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:14:47.927918 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608e10b-7bdf-4158-9164-85e9102aff86 service placement] REQUEST_METHOD: `GET` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:14:47.928230 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608e10b-7bdf-4158-9164-85e9102aff86 service placement] SCRIPT_NAME: `/identity` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:14:47.928508 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608e10b-7bdf-4158-9164-85e9102aff86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81476) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:14:47.930258 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.fernet_utils [None req-6608e10b-7bdf-4158-9164-85e9102aff86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81476) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:14:47.941891 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608e10b-7bdf-4158-9164-85e9102aff86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:14:47.943447 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608e10b-7bdf-4158-9164-85e9102aff86 service placement] RBAC: Authorization granted {{(pid=81476) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:14:47.954214 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: [pid: 81476|app: 0|req: 1457/2915] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 24 09:14:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:14:57.949982 dv-cinder-835384-23-iscsi devstack@keystone.service[81476]: DEBUG dbcounter [-] [81476] Writing DB stats keystone:SELECT=89 {{(pid=81476) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}